Determining Validity of Research Framework

Determining Validity of Research Frame temptCHAPTER SEVENFRAMEWORK cogent evidence7.1Introduction This chapter foc traffic patterns on the military rank and validation of the example. The explore inclemency and method deployed to maximise the validity will be discussed in the first part of this chapter. Also, the unfailing fashion model validity will then be discussed. The second part of this chapter will analyse and discuss the results obtained from the paygrade of the frame acetify. The example evaluation has been carried out by academics and fulls practiti unitaryrs and has rated the suitableness and intensity level of the fashion model in the locution sector.7.2 ValidityThe validation process started at the first stage of the look, making current that the research participants, measuring instruments and context were exclusively accomplishing the research aim. A number of writers have contended that validity plentynot be applied to qualitative research. At the same time, have dumb the film for some qualifying check or measure for their research (Golafshani 2003). Clearly there ar no direct experiments validating qualitative research however, this does not imply that there are no guidelines (Patton 1990). Through the stages of this research, the effort has been made to guarantee that (1) the measuring instrument is measuring the concept being referred to and no other concepts and (2) there is an accu compute measurement of the concept (Bailey 1994). The measuring instruments used in the wonder method were the converse questions also the method used in analysing the data collected and building the framework. therefore, it was important to ensure that completely research propositions were suck in and hard-hittingly measured. Also, the method of abbreviation required the provision of right inferences. soft research has often been portrayed by flexibility and uncertainty, and tension in the midst of creativity and rigour. Therefore at the interview analysis and the stages of framework building, a precise compromise was carefully considered (Pyett 2003). The analysis of interviews engaged a deductive process whereby the author was his most trenchant critic (Cook and Campbell 1976, p 229). This is known as inborn validity. However, in this study, an open view picture of validity was used whereby several(predicate) techniques enhanced the analysis and spirit of the go for of association capture and convalescence and prompted more true(p) and valid development of the framework (Golafshani 2003). These methods includedAn in-depth revision of the instruments used for data collection and the method of analysisCarrying out a peer polish up of interviews themes and methods of analysisCarrying out a fender study assessing the language of the questions definitions and terms clarity of the purpose of questions for interviewConstruct a descriptive and relational framework for the interview analysis to make sure there is the consistency of analysis criteria and accurate interpretation and inferences.7.3 External ValidityDeveloping a framework for the capture and retrieval of WLC knowledge in a construction visualise is base on a small number of cases, and does not correspond this framework post be suited and hard-hitting to other construction plaques practising WLC in the construction sector. However, researchers are also interested in the generalisation of the findings beyond the investigated cases (Schwab 1999). The justification for carrying out an in-depth interview is that people involved in a matured practice have knowledge that would somehow or another not be accessible to the researcher. Therefore, it is the prime(prenominal) of the understanding that is vital, instead of the number of respondents who share it (Wainwright 1997).While researching a few cases may accomplish excellent validity by giving a fundamental understanding of the practice in those cases they have been broadly criticised as lacking external validity (De Vaus 2001). The generalising of findings to or across a target population ideal is known as external validity (Pedhazur and Schmelkin 1991). There are no efficient or verifiable methods to carry out external validation based solely on a single examination of a research relationship (Schwab 1999). However, a definitive way to assess the validity of the proposed framework is to test such principles in reality, yet within research practice, this is rarely conceivable (Pyett 2003).Framework testing by the application method, it could take quite a eyepatch before the reasonable judgment of the framework validity could be reached. Regardless of the possibility that this was to take a short time, there would be a wishing to access to a huge sample (representing the industry) and the framework implementation in this sector. However, in this particular case, this was viewed difficult. Nevertheless, a few other methods were able to ampli fy the framework validity in this study as outlinedSelect example case studies governings (of authoritative and beat out practices) from the target population in the interview stage.The pilot study provided a solid ground for understanding the current practice of KM and indeed it was utilised as guiding principles in using the best practice to suit the industry, which boosted the generalising or external framework validity.Looking for the evaluations of the suitability and effectiveness of the framework in the construction sector by experts practitioners and academics.7.4 Framework evaluationTesting the general framework feasibility by applying it to a large number of organisations representing the construction sector have been difficult in this research. The framework evaluation by experts was viewed as another alternative for evaluation. in fact, this method was deemed as take over with regards to risk, finance and value.Assuming that 100 randomly selected organisations are a vocalization sample and the resources required are accessible in those organisations, it would take six months for the testing and implementation of the framework (practically it would take years). However what if the framework did not work? This would mean lost time of 600 months (100 organisation X 6 months) and therefore significant effort. Therefore, in this case, the evaluation method by experts could be deemed the most appropriate method even though the generalise by implement method was deemed possible.The reason for the evaluation method accordingly was not to anticipate that the framework was suitable for the organisation of the participated experts to obtain the experts evaluation in regards to the frameworks suitability in the industry. The suitability of the framework was not seen adequate however because the suitable framework did not necessary means it would add value. Therefore, effectiveness was another criterion the evaluation sought to answer.7.4.1 ParticipantsJ ustificationBased on their go, the suitability and the effectiveness of the framework can be judged by experts by using the right tool techniques, required resources and methods. The time and effort required for the framework implementation can also be estimated and matched with the current practice.The participants involved in the framework evaluation is made up of both groups of experts academic and practitioners. The practitioners are involved with WLC practice in the construction ascertain and therefore have a better understanding of the daily work. Also, their participation in a project which requires WLC practice means they can work with other organisation who practice WLC in the construction sector and can understand the verity of work environments.Also, academics are expected to have a deep understanding of different methodological viewpoints, as they know intimately observation and analysis. hike, the academics rely greatly on the construction industry as a major sourc e of information, so does not imply that they will be isolated. Also, many academics are involved in the practice.SampleA total number of sixteen experts were divided as between the academics and the practitioner who participated in the framework evaluation. The participants year of experience ranges from octet to thirty-eight years, accounting for the total of three hundred and thirteen years of experience. The eight practitioners were from the three case study organisations who participated in the pilot and main interview of the study. The focus on practitioners was on those in the field with experience in WLC practice because they had an in-depth understanding of the WLC practice. Also, participating academics were from four universities in the UK, working in the built environment and information and knowledge management departments.ExpertsJob titleYear of experiencePractitionersOrganisation AProject theatre director35Quantity Surveyor20Building Surveyor7Organisation BQuantity Surveyor6Project Manager33Organisation CCost Estimator15Project Manager27Construction Manager7AcademicsSenior Lecturer12Senior Lecturer30Lecturer6Lecturer7Lecturer25Lecturer25Reader31Reader21Table 7.1 Characteristics of participating experts in the framework evaluationBecause the framework evaluation required a seminar presentation, this process was thought to be long, and it was expected a first rate of response. Therefore, it was trenchant to reach the target of participants by sending invites to a member of staff with the three case study organisations. The total number invitations forwarded was ccc of which 120 were sent to academics and 180 to the three case study organisations.7.4.2 Method of EvaluationEmails was used a tool to invite experts and includedAn overview of the research projectRequest to participate in the evaluation of the framework for the purpose of validityRequest on the best times which suits each of the case study organisation for a 40mins seminar presenta tionThe framework was presented to each case study organisation using Microsoft Office PowerPoint 2007. The presentation was interactive in which participants lessened the explanation charm diagram parts were moved in each presentation slides accordingly. The seminar presentation was arranged into ten slides which took a narrative sequence of logic since it began by clarify the parts of the framework independently and afterwards presented the strategic framework.Figure 7.1 A screenshot of the PowerPoint presentationThe questionnaire was designed and hosted using the online service provider Survey Monkey and consisted of two parts. Firstly, details about participants including name, organisation, job title, business and size of the organisation.secondly, the framework evaluation and this was included in two sectionsScore based questions to the rate the suitability and effectiveness of the developed frameworkSeeking the positions of the participants about the advantages, limitation s and ways of improving the framework.It was decided to present the framework diagrammatically showing the strategic, tactical and cognitive operational levels. The presentation gave a more in-depth description however diagrams were sent to four research students to assess their readability, and the outcomes affirmed that such diagrams could convey the concept and details of the framework if the reader had broad experience in construction management. One of the research students who took part in the pilot study was a medical doctor, dissociated from the construction industry, who found the framework unreadable, Although he has a good understanding of the major concept. Respondents were also sent an attachment of the presentations slides via email.7.5Results 7.5.1Suitability of frameworkParticipants were being asked to show their pass judgment for suitability for each of the parts of the framework where 1 implied unsuitable and 5 suitable. The answer from the analysis uncovered t hat the levels of suitability for all approaches were considered far above the ground.As shown in Table 7.2, 3.74 out of 5 is the lowest mean value for the architecture of the knowledge base. Although it is not indicated in the value that the architecture of knowledge base is a 100% suitable, rather, it shows that it is likely pass to be more suitable than unsuitable. The circumstance with regards to knowledge capture, retrieval and structure approach much better as they all have a sum up of 4 out of 5. The skewness measure has a negative value which shows that the mass of distribution concentrates on the side of suitable. companionship CaptureKnowledge retrievalKnowledge StructureArchitecture of Knowledge BaseNValid16161616Missing0000Mean 3.93754.06254.00003.7500Std. Deviation .85391.85391.894431.00000Skewness -.605-.863-.639-.343Table 7.2 Suitability of the framework componentsFigure 7.2 shows that the majority of the participants gave a 4 or 5 rating of the suitability of all t he framework components. With regards to the architecture of the knowledge base, the suitability was rated under 3 by two participants (i.e. 2). The normal curve illustrates that the approximation of the average of all components some falls in the 4 rate of suitability. All the indications provide evidence that those components are suitable for the construction organisations.7.5.2 force of framework Participants were being asked to show their rating for effectiveness for each of the components of the framework where 1 implied to be ineffective and 5 effective. The mean value of effectiveness for all components as shown is (Table 7.3) are above 3.75 and ranges between 3.87 and 4.18. The skewness measure has a negative value which shows that the mass of the distribution is concentrated on the side of effectiveness.Knowledge CaptureKnowledge RetrievalKnowledge StructureArchitecture of Knowledge BaseNValid16161516Missing0010Mean4.06254.187503.93333.8750Std. Deviation.85391.910586.79881 1.02470Skewness-.863-1.019-.842-.571Table 7.3 Effectiveness of the framework componentsAs shown on the histograms shown in Figure 7.3 shows that the majority of experts participants evaluated all components to a score 4 or 5 rates of effectiveness. Knowledge retrieval thus turns out slightly more effective than the other framework components. The normal curve makes it clear the approximation of the average rate of all components which almost lies in the 4 rate of effectiveness. All the indicators together provide evidence that those components will be effective when implemented.7.5.3The whole Framework SuitabilityExperts participants were asked to indicate their rate of suitability for the entire framework using the same criteria as mentioned above. As shown in Table 7.4, the mean value of suitability (3.93) and the negative value of the skewness measure provide an encouraging indication for the external validity. In Figure 7.5, the pie chart shows that the majority of experts part icipants (62%) rate the suitability 4 out of 5 and the vast majority (81.3%) are either giving the framework a rate of 5 or 4. Furthermore, the histogram in Figure 7.4 shows that only one expert gave a rating of suitability under 3 (i.e. 2). Therefore, the indications of suitability are positive and show that this framework is considered suitable.The Entire Framework SuitabilityNValid16Missing0Mean 3.9375Std. Deviation .77190Skewness -.881Table 7.4 The entire framework suitability7.5.4Effectiveness of the Entire Framework Expert participants were asked to indicate their rate of effectiveness for the entire framework using the same criteria as mentioned above. The mean value (4) and the skewness measure value (negative) which shows that the framework is evaluated to be effective as shown in Table 7.5. A third of the experts participant shows a full mark of effectiveness, and 40% scored the framework a rate of 4 for effectiveness as shown in Figure 7.7. However, as shown in Figure 7.6 , three experts rate it a 3 and one a 2. Nonetheless, the normal curve is pointing to an approximation of effectiveness at the rate of 4. Indications show that the framework is effective.Effectiveness of the Entire FrameworkNValid15Missing1Mean4.0000Std. Deviation.92582Skewness-.623Table 7.5 Effectiveness of the entire framework.7.5.5Experts Opinion about the FrameworkThe expert participants were requested to express their opinion about the framework regarding suitability, effectiveness, advantages and limitations. This could give additional assessment not covered in the closed-ended questions. Twelve of the feedbacks were received and had been listed belowIt looks good to me. The problem is getting it work in practice.Great model and l like it, particularly the knowledge forms and the categorisation. This will help save time when searching for knowledgeI appreciate the overall seminar presentation was good and it covered a lot of the things we do not consider as worth pointing out in our sidereal day to day project delivery implementation.It is very hard to make a judgment on these key issues just on the presentation. However, I am quite shake by the approach to knowledge capture. Although the presentation is great, however, the information, it provides is insufficient.It seems very general.The framework works well in suitability and effectiveness and in use of collaboration tools, intranet tools with knowledge management within organisations. It is essential for a large organisation that staff can find and utilise answers as well as gaining access to the theme matter, experts and sharing information on similar projects and programmes. The advantages are that knowledge is shared rather than lost or protected by individuals. The saying that knowledge is government agency is true of individuals However in organisations that are able and willing to capture knowledge and share this with the wider team rather than isolating it with individual project members. The framework is suitable and effective in many bailiwicks and an insightful and logical structure. There is some duplication, but that would seem to be unavoidable.Leadership support is important, with top-bottom commitment. Co-operation/sharing is not always a natural inclination in individuals.The framework seems to be highly advanced and would require an IT system to enhance its processes and to encourage people to use it. If all members of the organisation are contributing positively to the framework and most of them have the ability to capture, retrieve, and then re-use the captured knowledge, then the system will be for sure highly effective. In reality, I think we share the knowledge sometimes, but even when we do so, we do not document or capture them properly. As a result, we lose whatever we have shared as we cannot capture, structure and re-use all the knowledge captured by only relying on our brain capabilities. We as practitioners need such a framework to assist us to have a better understanding of the knowledge so we can use it appropriately. From an overall point of view, I admire you for this excellent presentation, and it was informative and well structured. We have learned a lot from this insightful presentation about knowledge management. There is no doubt that this framework is needed for business success.Although the framework appears to be complicated on its operational side I am certain that if the KM operation depends on this model, the result will be relevant and highly efficient.The framework makes sense from a project management perspective. However, knowledge capture and retrieval are complex and resource intensive activities and can be onerous to the project team if proper incentives are not aligned with the leaderships expectation of knowledge sharing.I think if your framework is properly applied, it will be extremely effective. Also, it is highly suitable. Regarding its advantages, I think it is simple to read and understand by any psyche who has a basic level of knowledge management and its management.The issues mentioned in the comments can be grouped into four categories Firstly, those which highlight the role of knowledge capture and retrieval.Secondly, those who stock the importance and need for such a framework, due to the awareness of the problem, and thus confirm the motives for this research.Thirdly, indications that the framework may require a huge effort and construction projects already involved in demanding work. This issue has been discussed in the statement of the problem (section 1.4) whereby the problem lies on the capture and not the retrieval side. To minimise the problem, the project review was adopted as the main technique for knowledge capture which already exists in projects, rather than introducing a new approach. Additionally, the daily based reviews were rejected in this research for the same reason. Furthermore, a strategy for motivation has been pinpointed in the framework to ensure the application of knowledge capture.Fourthly and most importantly, a suitable and effective framework will also require careful application and effort. This has been said by several experts, and therefore considered a critical point, because the good theory does not necessary guarantee successful application.Finally, an expert described the framework to be very general. As aIn fact the framework is considering a particular area to investigate, related to knowledge capture and retrieval of project management processes, in the context of the project. Models inside the framework have provided precise details regarding what to do in particular scenarios. Also, knowledge capture and retrieval cannot be tackled in isolation from issues such as.Leadership support and cooperative culture. It is worth mentioning that this expert selected the rating of 2 for all of the 10 rating-based questions which are considered statistically biased. However, it was thought that this academic expe rt provided his personal details and these matched the tracking information. Therefore it was considered a valid response, and his opinion was appreciated. otherwise points were also considered related to the advantages of the framework and impressions about the presentation.7.5.6 Framework ImprovementAnother question was put forward in regards to how the framework can be improved. The feedback from the expert participants are listed belowBy trial and error.The framework can be improved after applicationl did not analyse the presentation critically to make rational input in this directionI would rather read the whole PhD thesis and make my suggestionsWhat issues should be addressed? Profitability, time and teamwork?There is no such thing as ideal knowledge framework. Therefore the knowledge framework can also be improved. Evolvement of a framework can be made by lessons learnt from previous projects receiving constructive criticism and using good management techniques such as benefi ts and stakeholder management. Furthermore being clear about the benefits of knowledge capture and retrieval, with the onward transmittal/access of results by stakeholders, communication and using the framework communication channels for optimum use. The framework is excellent for understanding the needs and requirements of this age, and that knowledge should be shared for the benefit of all within an organisation rather than being kept and possibly lost by the few, when the project team is dissolved or re-assigned.Show an example of how it will operate in practice.Working hand in hand with the IT system which enhances its process and records all captured knowledge after structuring it.The answers can be categorised into four major groupsFirstly, improvement is a continuous process, and the most appropriate way to improve the framework is through the application.Secondly, improvement can be achieved by addressing time, profitability and teamwork which have already been addressed in this study. Profitability is a significant factor that can declare the importance of knowledge capture and retrieval of the project and is, therefore, one of the areas to be embarked upon. However, the development of a model to measure economic profit unsocial is enough to do a research project.Thirdly, the exertion of effort in linking the IT system with knowledge capture and retrieval approaches. The knowledge base is a component of the framework, and there is a need to train people on the use of the knowledge base which links knowledge capture and retrieval approaches.Finally, the use of labels to love between compulsory and optional activities and resources. This has been presented differently in this research as the majority of the activities and resources were thought to be important if knowledge was required to be captured and retrieved successfully. It was decided only to emphasise activities or resources that were suggested but not deemed necessary.7.6Summary This chapter has set out to determine the validity of the framework. It has demonstrated that validity relies on of the inherent validity of the methods of data collection, analysis, and the inferences led to the construction of the framework. Critical in-depth revision, piloting, peer reviews and a fixed framework for analysis were the methods used in this research to maximise its validity.For the purpose of generalising (external validity) the framework, a number of factors were considered early on in this project. These included using a randomly selected sample in the survey choosing a sample of best practice from the target population and comparing the results of best practice with the current practice obtained in the survey to assess the ability of and utilise the solution for the target population. Further steps were taken to test the generalisability of the framework by inviting practitioner and academic experts to evaluate its suitability and effectiveness. The results have indicated th at the framework and its components are applicable and effective.