Instructional strategy considerations

What are the most important considerations when choosing a particular instructional strategy?

1. learning objectives - For example, if my learning objectives focus on learning a procedure, I would not choose Problem-solving instructional strategy. If learning objectives state learners are to learn skill to make something, I'm going to use something like Shank's Learning by Doing theory. As part of this, learning objectives need to be able to be categorized in a learning taxonomy like from Tripartite (Hilgard 1980), Gagne (1985), Bloom (1956), Anderson (1981), Merrill (1983), Reigeluth & Moore (1999), etc.

Is it important for students to drive the speed of the course, are experience necessary for the experience, are teacher-directed methods appropriate?

2. desired/prescribed learning environment - setting, use of simulation, importance of hands on experience, configurations
3. available resources - time, personnel; if there is limited time, consider applying teacher-directed strategies
4. learners/target audience - do learner characteristics really change the instructional strategy? 2c argues no. It might affect the delivery mode, though. Checklists are most appropriate for procedures with one answer. Higher order thinking with multiple correct answers are best to use with assessment rubrics. Novice versus expert - novices might need a more structured learning strategy where they have lots of scaffolding and have it slowly taken away.

Types of tests

Entry test - prerequisites Pretest - adapt instruction based on results Practice tests Final test

• normative referenced
• ISD doesn't do this because it doesn't measure skills; psychometrics
• critereon referenced
• conventional - m/c, t/f; good for assessing verbal outcomes, some concepts and principles
• performance/product checklist - procedural learning; one way to do something, one way of learning
• rubrics - multiple ways of doing something

Comparison of Alternative Instructional Design Models

by: Marlene Fauser, Kirk Henry, and David Kent Norman on February 4, 2006

An Instructional Design model gives structure and meaning to a problem, enabling the would-be designer to negotiate the design task using a process or systematic method. Models help us to visualize the problem, to break it down into discrete, manageable units. The true value of a model can only be determined within the context of use. A model should be judged by how it mediates the designer's intention, how well it can share a work load, and how effectively it shifts focus away from itself toward the object of the design activity (Ryder, 2006).

The purpose of this paper is to analyze three instructional design models, one from each of the classifications classroom, product, and system. The analysis should compare and contrast the key concepts associated with each model. Additionally this paper will identify and describe arguments against the use of ISD models in general.

Instructional Design models are classified into three types, classroom, product and system (Gustafson & Branch, 2002). Classroom models are of interest to, and are usually designed for, professional teachers from K-12, community colleges, vocational schools, and other related areas. These models take into consideration the environment of teachers. Their users may view them as a guide rather than a methodology. These models focus on the use of existing materials rather than the design of new. The output of these models is small, a unit or module of instruction used within the school year. The models also have less rigorous formative evaluation and revision than product or system models.

Product-Oriented models, prescriptive in nature, are primarily focused on creating instructional products. These instructional products may be self-study, self-paced computer based training, or other materials that can be used by a student with reduced guidance. The methodology of these models may cause them to be confused with system models, but product models are not used to create comprehensive systems as system models are. Product development models demonstrate four assumptions: the instructional product is needed, something needs to be produced rather than using or modifying existing materials, there will be considerable emphasis on tryout and revision, and the product must be usable by learners with only facilitators but not teachers. These models are drawn upon as computer-based instruction has become more often the instructional delivery method of choice (Gustafson & Branch, 2002).

Systems-oriented models are used to develop large amounts of instruction. This can be courses or curricula and may include the development of new materials or the repurposing of existing materials. Systems models align with the ADDIE methodology, emphasizing front-end analysis and design phases. Gustafson and Branch observed emphasis on front-end analysis may result in a proposal for a non-training solution (2002). Systems models, unlike product development models, typically assume a large scope of effort.

Classroom Oriented Model - Gerlach and Ely

The Gerlach and Ely Model is a prescriptive model that is well suited to K-12 and higher education. It is meant for novice instructional designers who have knowledge and expertise in a specific context (Braxton, Bronico, & Looms, 1995). Although drawn as a linear model, many of the steps are intended to occur simultaneously. The Gerlach and Ely Model recognizes most curriculum will be designed around the concepts to be taught in each subject matter and teachers, who are forced to adapt existing materials for use in their courses (Braxton, et al., 1995).

The first step in the Gerlach and Ely process is to simultaneously specify the content and objectives. With this model intended for K-12 teachers, these combined steps recognize that teachers know the content or think of content first, then define the objectives for that content. The objectives do need to be defined, however, for use in the next steps of the model.

The second step is to specify the entry point of learners, which for K-12 teachers may simply mean reviewing existing records. The third step is to perform five activities simultaneously: (1) determine strategy, (2) organize groups, (3) allocate time, (4) allocate space, and (5) select resources. The key is to determine which combinations will best allow the students to meet their objectives. The Gerlach and Ely model emphasizes using existing instructional materials rather than develop new materials.

After these simultaneous decisions are made, the next step is evaluation of performance. The teacher should determine what observable or measurable changes occurred in the students and if the observations can be attributed to the instruction. The model concludes with a feedback loop to determine the effectiveness of the instructions so changes can be made as necessary.

Product Oriented Model - Bates

In 1995, Tony Bates presented a model for developing open and distance learning. Bates realized digital communications were going to be used more frequently for distance education. Early in his research, he acknowledged the fact that there are inherent limitations of distance education. Bates created a framework for selecting learning technologies which is still relevant today. The organizing framework that Bates proposed is called ACTIONS, which stands for access, cost, teaching and learning implications, interaction, organizational issues, novelty and speed.

Bates's model has what he calls a front-end system design. A front-end system design has four phases: course outline development, selection of media, development/production of materials, and course delivery. Within each phase, Bates has properly identified the team roles and the actions or issues to be addressed. Bates readily admits he borrows heavily from the ADDIE model from which he derives many of his ideas. One of the reasons Bates may have felt compelled to create an additional model to ADDIE is because ADDIE has been widely and often criticized for being too systematic. ADDIE is often said to be too linear, too constraining, and even too time-consuming to implement with regards to e-learning. While Bates may have been making an attempt to simplify ADDIE, he cautions course design may take as much as two years.

One of his main concerns, prompting the development of ACTIONS, was the way face to face course material was adapted for web or other distance learning approaches. Bates feels the traditional remote instructor concept is nothing more than face-to-face instruction without direct interaction. Bates also states these specific scenarios often fail to take advantage of the unique benefits that are available through the specific technology being used.

Additionally, in his 1995 book, Technology: Open Learning and Distance Education, Bates details cost analysis when designing distance or e-learning solutions. If his design and evaluation methods are followed, one would easily be able to determine the effect each dollar spent on this e-learning solution has had on a given student. This can be particularly useful when dealing with fiscal management issues.

The Bates Model provides the user with a plethora of tools and ideas, each of which will be useful for designing e-learning environments. The model is broken down into four distinct groups. Each group is replete with projects and assignments.

System Oriented Model - Gentry IPDM

Castelle Gentry explains his model of Instructional Development in the course of a textbook. Gentry named his model the Instructional Project Development and Management (IPDM) Model. The IPDM model was born as a result of five conclusions Gentry made with regards to instructional design. The first conclusion admonishes introductory instructional design textbooks for not adequately linking relationships between instructional design processes and their supporting processes (Gentry, 1994, p. ix). The second of Gentry's conclusions says introductory instructional design textbooks should teach both what and how to "do", in other words "practical means for accomplishing specific tasks" (Gentry, 1994, p. ix). The third through fifth conclusions set a basis for Gentry to develop a generic Instructional Design model as a survey of instructional design processes, supporting processes, and techniques.

The result IPDM Model is has eight development components, defined by Gentry (1994) in a non-linear diagram.

1. Needs analysis - establish the validity of needs and goals for existing or proposed instruction
2. Adoption - establish acceptance of an innovation by those affected and obtain a commitment of resources.
3. Instructional Design - determine and specify objectives, strategies, techniques, and media for meeting instructional goals.
4. Production - construct elements of a project, as specified in a design and revision data.
5. Prototyping - assemble, pilot test, respecify, validate, and finalize an instructional unit.
6. Product Installation – establish the necessary conditions for effective operation of a new instructional product or process.
7. Ongoing Operation – maintain the continuing application of an instructional product and/or procedure.
8. Ongoing Instructional Unit Evaluation – collect and analyze data about an ongoing instructional unit to make decisions about future revision.

The development components have five supporting components, defined by Gentry (1994, p. 5):

1. Project Management – control, coordinate, and allocate resources.
2. Information Handling – select, collect, organize, store, retrieve, distribute, and assess information required by an ID project.
3. Resource Acquisition and Allocation – determine resource needs, formalize budgets, and manage resources.
4. ID Project Personnel – determine needs for staffing, training, assessment, motivation, counseling, censuring, and dismissing ID project members.
5. Facilities – organize and renovate spaces for design, implementation, and testing of elements of instruction.

The development and supporting components are linked by information sharing between the two clusters for the duration of the project. Gentry designed his model to show instructional design is not a linear process.

Model Comparison and Contrast

When doing the comparison between Bates, Gentry and Gerlach/Ely's models, it was noted that all three models allowed for some variation in implementation; it would not be incorrect to modify the order of the steps depending on the settings of the instructional situation. This is especially true for the Gerlach/Ely model. The three models we have chosen to review all were published between 1994 and 1995.

The Bates and Gentry models both rely heavily on front end work. This means that the model assumes that step one will involve an extensive needs analysis. The information learned from the analysis will be crucial in the next phases of the design. In particular, the Gentry IPDM model is best for large scale projects, as demonstrated by the communication core of the model. Both Bates and Gentry also provide a framework for helping the designers determine the overall cost of the solution. This may include the cost of the technology needed for implementation. Bates and Gentry are also very good at providing personnel assignments for each phase of the design. This allows the designer to determine, up front, the type of personnel that will be needed throughout the design and implementation.

The main strength of the Gerlach/Ely model is practicing classroom teachers can identify with the process it suggests. The Gerlach/Ely model allows for a novice instructional designer whereas Gentry and Bates need some design expertise to perform front-end analysis. As a result, classroom teachers might be more likely to apply Gerlach/Ely. The Gerlach/Ely model differs from the other two by emphasizing existing content as the basis for new instruction. Basing new instruction on old content by forgoing front-end analysis may unintentionally reinforce traditional learning teaching patterns rather than promoting a re-examination of best practices in classrooms.

Instructional Designers cannot be effective if they are familiar with only one model.  The designer must be able to fit the design to the situation and familiarity with various models will make that designer more successful.  Analyzing various models demonstrates that although the models had differences, combined steps in various ways, or used different vocabulary, they shared a fundamental principle of attempting to deliver effective learning or educational tools.

Arguments Against Instructional Design

In an article published in Training Magazine in April 2000, Jack Gordon and Ron Zemke lay out arguments against the use of ISD models. The article was the catalyst for a plethora of rebuttal articles, white papers, and seminars on why ISD is not dead and is still useful and relevant to the field (Clark, 2004). The arguments in the original article were:

• ISD is too slow and clumsy to meet today's training challenges
• There's no "there" there
• Used as directed, it produces bad solutions
• It clings to the wrong world view

Rebuttals concentrated on the question: Is it ISD that's flawed, or the manner in which it is applied that is the problem (Zemke & Rossett, 2002)? To address the argument of ISD being slow and clumsy, Sivasailam "Thiagi" Thiagarajan conducts workshops in "Rapid Instructional Design" which includes strategies and design elements to move more quickly through the ADDIE phases and to use partial processes where appropriate (Thiagi, 1999).

Argument two discusses the use of ISD as a "technology" of instruction. Project management for Information Technology (IT) projects is seen as a science. Yet IT project management is an art as is Instructional Design. IT project management has addressed similar flaws in waterfall or linear project management processes. Additional project life cycles have been introduced in this discipline. These models, selected based on project and environmental factors, include: incremental or prototype based methodology, Barry Boehm's Spiral Model, and the Rush to Base Evolutionary Model (BU, 2005). Options such as these applied to ISD would allow for the flexibility and rapidness desired for delivery of an instructional solution.

The third argument is that following ISD models will produce bad training. Clark (2004) discusses how the "A" of ADDIE was never meant to stand for Performance Analysis and that Instructional Design should only begin once it is determined training is the solution to the problem.

Charge 4 assumes ISD jobs have a set of best practice procedures that must be taught to students who are not master performers. Saul Carliner suggests we should remember ISD is a value system that must be applied appropriately and all ISD needs is re-tooling for all the new ways of learning we see since ISD was first proposed more than 50 years ago.

References

Boston University Corporate Education Center. (2005). Courseware MDP405: Managing Information Technology v4.0.

Braxton, S., Bronico, K., & Looms, T. (1995). Instructional design methodologies and techniques. Retrieved January 30, 2006, from University of Michigan, Educational Software Design and Authoring Web site: http://www.umich.edu/~ed626/Gerlach_Ely/ge_main.htm

Carliner, S. (2003). An instructional design framework for the twenty-first century. Unpublished manuscript, Concordia University. Retrieved January 30, 2006, from http://education.concordia.ca/~scarliner/idmodel.pdf

Clark, D. (2004, June 7). The Attack on ISD – 2000. Retrieved January 30, 2006, from http://www.nwlink.com/~donclark/history_isd/attack.html

Clark, D. (2004, June 13). A hard look at ISD - 2002. Retrieved January 30, 2006, from http://www.nwlink.com/~donclark/history_isd/look.html

Gentry, C.G. (1994). Introduction to instructional development: Process and technique. Belmont, CA: Wadsworth Publishing Company.

Gordon, J. & Zemke, R. (2000, April). The attack on ISD: Have we got Instructional Design all wrong?. Training Magazine.

Gustafson, K.L., & Branch, R.M. (1997). Survey of instructional development models: Fourth edition. Syracuse, NY: ERIC Clearing house of Information & Technology

Ryder, M. (2006). Instructional Design models. Retrieved January 29, 2006, from University of Colorado at Denver, School of Education Web site: http://carbon.cudenver.edu/~mryder/itc_data/idmodels.html#comparative

Thiagarajan, S. (1999). Rapid Instructional Design. Retrieved January 29, 2006, from http://www.thiagi.com/article-rid.html

Zemke, R. & Rossett, A. (2002, February 1). A hard look at ISD. Training Magazine

Analysis Techniques Portfolio

by: Marlene Fauser, Kirk Henry, and David Kent Norman on February 18, 2006

Task Analysis involves the systematic process of identifying specific tasks to be trained and a detailed analysis of each of those tasks (Ehrlich, 2000). To be able to design learning supported instruction, designers must understand the tasks the learners will be performing (Jonassen, Tessmer, & Hannum, 1999).

The purpose of this paper is to apply three analysis techniques from separate categories of methods as described by Jonassen, et al. to one instructional goal. This paper describes the key concepts, application, and experiences with a technique each from the Job, Procedural, and Skill Analysis Method, Instructional and Guided Learning Analysis Method, and Subject Mater/Content Analysis Method. A sample of each analysis technique is provided in the Appendices.

The Job, Procedural, and Skill Analysis Method emerged during WWII when most training being developed was technical training which emphasized procedural skill development. These methods describe in a procedural manner the way jobs are performed. Analyzing steps has become the most common conception of task analysis and there are many methods and approaches published. Alternatively, Instructional and Guided Learning Analysis Method describe tasks how they are best learned. This may or may not be consistent with the ways the job is performed. Subject Matter/Content Analysis Methods consider that designers elicit instructional information from SMEs. This information will usually come to the designer in an outline form, chunked by the subject matter expert. This organizational method can be effective, but is not always the best way to represent the instructional content. Subject Matter/Content Analysis Methods offer alternate methods for representing content structure.

Job, Procedural, and Skill Analysis Method – Job Task Analysis

The task analysis approach emerged from the field of systems analysis when systems analysis was applied to the world of work (Jonassen, et al., 1999). The purpose of creating task descriptions is to enhance the performance of the overall system or to enhance job or task performance.

In order to conduct a job analysis, you must describe and analyze each task. The description should include the stimulus situation and the proper response. One would typically start at a general level and then get more specific (Miller, 1962 as cited in Jonassen, et al., 1999). Jonassen, et al. would argue that there are several advantages to creating task descriptions:

• It is very job oriented
• It provides data to support training, personnel selection, and job design
• It describes tasks in sufficient detail to prevent misunderstanding
• It identifies the competencies that underlie job tasks

We must also consider the disadvantages of task descriptions as well. First, the process is generally limited to specific descriptive aspects of tasks. Secondly, the skill level of the task analyst must be high. Individuals with this level of skill may be difficult to find. Additionally, it has been found that the task description procedure as a whole does not help select specific tasks for training.

All of this information was considered when creating the analysis for the task of distance collaboration. The outline in Appendix A contains detailed information that follows the methods suggested by Jonassen, et al. (1999). There is additional information in Appendix B that further develops each solution and what it should be used for.

When going through this assignment, it was difficult for me to understand the level to which the analysis should be reported. I was torn between describing each button push and mouse click. For our chosen area, there are several categories where different applications could be used. I felt that it would be difficult to describe each button push for each individual software package as they did in the book for pre-flight checks.

I did notice, however, that putting this information on paper, provided me, an experienced technologist, with clear concise information which can and will be used to help others decide on platforms as well as conduct smooth, effortless collaboration sessions.

Instructional and Guided Learning Analysis Method - Learning Contingency Analysis

Learning contingency analysis differs from task analysis primarily in focusing on tasks in a learning environment rather than job performance (Jonassen, et al., 1999, p. 99). By focusing on identifying the behavioral components of tasks, the designer can discover the interdependencies among the tasks (Jonassen, et al., p. 99). Contingency analysis involves first sequencing the instructional content, followed by determining the most effective conditions for the instruction. For example, teaching algebra to a six-year-old during recess would be an extreme example of poor contingency analysis. First the child must have an understanding of basic arithmetic and a learning environment free of distractions.

The sequencing of instruction should reflect one behavior having dependence on another (Jonassen, et al., p. 100). To sort learning contingencies, Jonnassen cites Gropper's (1974) four behavioral components tasks can have:

• Superordinate/subordinate – a hierarchical relationship to show behavior X is a part of behavior Y.
• Coordinate relationship – behaviors at the same level in a hierarchical relationship, not necessarily performed at the same time or in the same order.
• Shared elements – behaviors that share some amount of the same concepts.
• No relationship – behaviors with no relationship to each other.

Appendix A references eight steps for conducting a learning contingency analysis, which includes Gropper's behavior components as part of steps three and four. The hierarchical learning analysis is similar to Dick, Carey, and Carey's (1999) chapter on subordinate skills analysis. Dick, et al, describes a method of drawing relationships between learning components in a pictorial form, versus a bulleted, tabular form in contingency analysis. The fifth step of contingency analysis skips a few chapters in the Dick and Carey model, but is analogous to developing instructional strategy.

In all, the learning contingency analysis is best for environments where learning is important, rather than performance. It is a time consuming analysis meant to draw a roadmap for the most effective sequence of instruction. The result instruction framework is rigid and may not be appropriate for learning tasks which can not be directly observed. A large instructional unit could make a confusing set of sequential dependencies between tasks in step 3 as compared to the pictorial method recommended by Dick, et al., so a highly skilled analyst is to perform contingency analyses.

Subject Mater/Content Analysis Method – Master Design Chart

The Master Design Chart approach to task analysis is used when the educational ends are the development of the behavioral capabilities of the students (Jonassen, et al., 1999, p. 207). This method has its basis in two foundations. The first foundational principle states that instructional outcomes can be stated as behavioral objectives. The second states the use of instructional taxonomies to describe the outcomes. In using this method, the instructional designer classifies content according to behaviors but is not bound to any taxonomy.

To conduct a Master Design Chart analysis the designer constructs the behavior axis using a taxonomy of instructional outcomes. The designer can use more specific behaviors as appropriate. Next the specific items of content are identified and listed as the vertical axis.

Each cell in the chart is now analyzed and the relative emphasis required in the course or curriculum is noted in the cell. This is done on a continuum with 0 representing no emphasis and a number such as 3 or 5 representing the most emphasis. Although the resulting chart does not represent relationships among the instructional outcomes, it is recommended a relationship step be taken.

A Master Design Chart is constructed for a curriculum or a whole course rather than for an individual lesson. Although it gives an overview of the entire course, no information is included regarding any instructional strategy. A strength of this method is that the risk of omitting important content is reduced. Weaknesses of this method are that it is dependant on the skills of the analyst, lacks basis in needs assessment and is time consuming to construct.

For our course, the matrix was created using document analysis techniques. The result is included as Appendix E. The content included information about being a "Distance Manager" and included content on what a leader should learn and know to successfully manage a geographically diverse team. Concentration from this SME was on higher level techniques as opposed to the detailed tasks analyzed in the Job Task Analysis approach.

While creating this matrix it was difficult to determine how the task list should be written. My habit is to include the action verb associated with the learning event. The matrix provides that association to the taxonomy, a challenge for this designer.

I selected to analyze this method since much of my work comes to me from Subject Matter Experts. I hoped I could learn to use a Subject Mater/Content Analysis Method in my job performance. I found it contrary to my method of thinking, which does helps me see things from a new perspective. However I also found it tedious and am not anxious to complete the Master Design Chart.

Overall Reflection

After reviewing the previous techniques, it has become clear that while some techniques can be used for the broadest of applications, others have been designed specifically for certain tasks. The more finely tuned the instructional designer's skill set is, the more they will be able to select the most specific tool for the task.

The task analysis method could be considered the broadest within the subset that we evaluated. It could be used in nearly every scenario. However, the resolution that it lacks may cause the end product to be deficient. Learning about and when to use the most refined tools available is the difference between being a novice or an expert.

References

Dick, W., Carey, L., & Carey, J. O. (2005). The systematic design of instruction: 6th edition. Boston, MA: Allyn & Bacon.

Ehrlich, D. (2000). Glossary of terms. Retrieved February 18, 2006, from the Northeastern Illinois University, Instructional Design II Web site: http://www.neiu.edu/~dbehrlic/hrd408/glossary.htm

Fazio, A., & Rieff, K. (2006, Jan). The distance manager. Presented at a meeting of the IT department of Walt Disney Parks & Resorts.

Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1999). Task analysis methods for instructional design. Mahwah, NJ: Lawrence Erlbaum Associates

Writing up reliability results

Guiding Principle: No measure is ever reliable. Scores determined by a measure may be reliable depending on who completes the measure and the conditions under which the measure is completed (when and where).

Structure of the Reliability write-up

Procedure One	Procedure Two
1. Comment on the reliability of the scores by examining the reliability coefficient. Everyone has her/his own standards, but here are what mine tend to be: < .65 Poor; .65 ≤ Modest ≤ 80; > .80 Very good	1. Comment on reliability coefficient
	2. Comment on the corrected item-total correlations
	Note which items have a negative corrected item total correlation. Note whether any negative corrected item total correlation are attributable to miscoding or items with response options are scaled opposite of the majority of items. Note which items have a zero corrected item total correlation. Note which items with a zero corrected item total correlation do so because of no variance (every person responded the same way to the item)
2. Do nothing more	3. Comment on how high your reliability coefficient would increase if you were to drop out all negative item-total correlations one at a time until you had no more negative corrected item-total correlations.
	4. Do nothing more

Sample paragraph

Responses for the various features of the agency website were judged to be moderately reliable for the customer service representatives who participated in the survey, with a reliability coefficient of 0.778. A review of the corrected item-total correlations suggests that the questions for the website as a critical tool and a unique website in the marketplace do not correlate with the corrected total very well. Their elimination is warranted on the basis that reducing the scale to only relevant items would make for a better, more parsimonious scale. It turns out that removing the item may further be motivated by anticipated increase in the reliability coefficient reported in the output (0.884).

If HTML newsletters, online payments, Internet as a tool, and customer communication with the agency were removed, the reliability of the scale would increase to 0.963.

To examine the impact of removing both items, each item was removed one at a time. This approach is necessary because the impact of removing one item changes the relationship of the other item with the changing total.

Developing a research question

Three major types of questions are causal/comparative, relationship, and impact. Use a literature review as a funnel to narrow a broad area of research into a manageable contribution, which leads to "The purpose of this study is..." Classic or landmark studies would be considered good content for the introductory, broad part of the literature review. Use incremental blocks of studies as a transition to more focused topics in your area of research. The final question should be something you are genuinely interested in finding the answer to so you have the motivation to complete the report as time progresses.

The question should have a question mark at the end. The question should ask if there is a cause/effect, what impacts what, or if there a relationship (what two or more things are related). Should have a construct or construct level at the start that is narrowed down to something very specific. Could "self concept" really mean "body image"? Does "achievement" mean achievement in math, reading, overall, or something else? Include operational definitions for parts of the question to define "achievement" if you are looking for relationship between achievement and yearly income. Perhaps achievement has several six factors to consider observing in the study.

Research problem statement template

Template:

This study will investigate the effects of (treatment) on (population). The investigation will be conducted (describe the setting). The present study, within the context of (the setting), will be concerned with the following research questions:

1. Does the procedure of (treatment) have (performance change) difference on (the experimental population) as compared to (the control population)?
2. Does the method of (treatment) - (restate method of treatment) - affect (population)?

Hypothesis 1: (Relate back to research question number 1). Hypothesis 2: (Relate back to research question number 2).

---

Example:

This study will investigate the effects of Digital Propensity on the preferences of insurance agency employees when information is presented with either text or graphics as the primary presentation format. The investigation will be conducted in a "real work" setting and will use a computer-based task. The present study, within the context of a work environment, will be concerned with the following research questions:

1. Does the presentation of graphics with text support more closely match the preferences of "Digital Natives" than the same content presented as text with graphical support?
2. Does the presentation of text with graphical support more closely match the preferences of "Digital Immigrants" than the same content presented as graphics with text support?

Sample Chapter organization of study

Chapter 1 of this study introduced the problem statement and described the specific problem addressed in the study as well as design components.

Chapter 2 presents a review of literature and relevant research associated with the problem addressed in this study.

Chapter 3 presents the methodology and procedures used for data collection and analysis.

Chapter 4 contains an analysis of the data and presentation of the results.

Chapter 5 offers a summary and discussion of the researcher's findings, implications for practice, and recommendations for future research.

Validity issues in research

Internal validity - how valid were the procedures that I followed in terms of the research method to obtain the information I need External validity - how generalizable is the information learned from a study

Internal validity

• History effect - did something recently happen to one of the sample groups that would make it differentiate from the other groups. For example, if you ask small children about their fears of ghosts, some of the children may report less fear if they are surveyed near Halloween.
• Maturation - when the study doesn't control for natural growth; in other words, the natural maturation of the group is the actual cause of improvement in the study instead of the treatment
• Testing - what are the effects of a pretest on a posttest; familiarity with the instrument impacts the results on the second test
• Instrumentation - is an issue when instrumentation changes during a study; for example, calibration of individual opinions in a observation study could have differences in results just because a one rater might think an event is a 5 on a 1-10 scale, and another rater might think the same event is a 7. The result grades on essay papers depending on the mood of the instructor. If the publisher comes out with a new version of a test and the previous test isn't available. Then you need to see how closely the two tests are correlated.
• Statistical regression - also known as regression to the mean; for example, getting a perfect score on the SAT only leaves the possibility of going down in score. The more extreme the results, the more likely each is to move towards the mean because of the error that put them in the extreme in the first place like guessing on questions that make the difference between a perfect score and a lower score.
• Differential selection - people that you pick are not representative of the population in the first place;
• Experimental mortality - differentially loose respondents; for example, you might not even know who you lost or picked in the first place. Loosing the extremes in both cases of a paired study could be the actual cause of change rather than the treatment
• Selection maturation interaction - as part of the selection, the sample actually has unknown differences that make the differences in the results instead of the treatment. For example, if teaching children basketball, while the children may start with similar free-throw ability, one child may improve faster than another simply because of superior hand-eye coordination, rather than the basketball instruction.

External validity

• Interactive or reactive effects of testing - example: giving a pretest to the people in the population sample may impact their performance during observation, so the pretest has an interaction effect that limits the generalizability of the results to the general population because the whole population did not take the pretest.
• Interaction and selection biases - the treatment only works for the selection, so can't be generalized; may be a result of the environment the selected group was performing in
• Reactive effects of experimental arrangements
• Multiple treatment interference - when you have more than one treatment, how do you know which one made the difference? Something like sequencing presentation of treatments may make a difference.

Bibliography

Campbell, D. T. & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Chicago, R. McNally.

Predicting the performance of interpreting instruction based on Digital Propensity Index score in text and graphic formats

Practitioners have proposed that Digital Natives prefer graphics while Digital Immigrants prefer text. While Instructional Design has been extensively studied and researched, the impact of the graphical emphasis in instructional designs as it relates to digital propensity has not been widely explored. Specifically, this study examined the performance of students when presented with text-only and graphic-only instructional formats. The purpose of this study was to test the relationship between Digital Propensity Index scores of individuals and their performance when interpreting online instruction. A sample of students from the population of a large metropolitan university received the Digital Propensity Index questionnaire, which is a measure of an individual's time spent interacting with digital media. Each student was randomly assigned varying formats of a computer-based instructional unit via a public survey. The instructional unit consisted of the DPI questionnaire and six tasks related to the Central Florida commuter rail system.

Participants were asked to answer the DPI questionnaire on a website by clicking on a link in an emailed invitation. Following the DPI questionnaire, participants were randomly assigned to one of two groups. Group One saw three instructional tasks shown in text and shuffled in random order. Each task was displayed on its own webpage. By submitting an answer to the task, the group progressed through the website to the next task. Group Two saw graphic tasks first, again, shuffled in random order. After the first three tasks, the groups swapped instructional formats to view the opposing group's initial questions. Participants were timed on how many seconds they spent reviewing each task. Each task had an assessment question to evaluate the learning outcomes of the instructional unit. Finally, the DPI score of the participant was matched with the time spent viewing each presentation format.

The findings indicate that DPI score had a statistically significant prediction of time spent navigating each type of instruction. Though the link between DPI score and time spent navigating instruction was statistically significant, the actual measurable time difference between navigating text and graphic formats was only a fraction of a second for each increment in DPI score. Limitations and potential future research related to the study are discussed as well.

Attachment	Size
Norman_David_K_200808_PhD_protected.pdf	3.3 MB

Instructional Treatment Plan

As a class assignment, I developed an instructional treatment plan for the following objective: Given a virtual working environment, avoid communication problems between virtual team members when working on a project.

We chose Nelson's Collaborative Problem Solving (CPS) model because team skills are a higher order thinking activity. Nelson says CPS is not appropriate for memorized or procedural tasks and a CRT item would be "waste precious instructional time" and that "CPS is most appropriate when there is not a single answer to a question or best way of doing something" (Reigeluth, 1999, p. 247).

CPS was also chosen because the collaborative element in a virtual environment forces the students to experience the very topic they're studying and writing about. Each of the process activities in CPS is a small assessment of the enabling objectives since the group can't move on to complete the assignment until they have normed, agreed on the problem, defined roles, etc. I thought the true assessment of the enabling objectives would be performed as part of the debriefing, reflection, and discussion at the end of the unit.

Many texts have the condition, behavior, and criteria/degree elements of objectives (Dick, Carey, & Carey, 2005, pp. 123-139; Kubiszyn & Borich, 2003, p. 82; Simonson, Smaldino, Albright, & Zvacek 2006, p. 129; Stolovitch & Keeps, 1999, p. 356) and that Hirumi added audience (2005, ¶ 6). We tried to follow the samples in Dick, et al. (2005, p. 139), for the condition/degree of assessing intellectual skills by specifying things like "at least five peer-reviewed journals" in the sample assessment rubric. The other assessment rubric items were modeled from Dr. Hirumi's performance criteria for this assignment and peer evaluation for the end of the course.

I believe I can ask the question, "could I observe the learner doing this?" (Kubiszyn et al., 2003, p. 80; Dick et al., 1999, p. 127) to each of the objectives we listed in the treatment plan as part of the measurable behavior component of the objectives. Granted, the observation would not have an associated percentage completion or accuracy, but that goes back to the complexity of multiple solution theory in CPS. Dick et al, p. 139 has several samples without a CRT item as the assessment criterion for intellectual skill objectives.

Ways to improve the treatment plan are to include some of the degree elements from the assessment rubric earlier in the objectives, add more description and content to the instructional events, create an activity for forming and norming teams and an activity for role selection, and investigate taking some fuzziness out of the objectives.

References

Dick, W., Carey, L., & Carey, J. O. (2005). The systematic design of instruction (6th ed.). Boston, MA: Pearson Allyn and Bacon.

Hirumi, A. (2005, August 1). Unit 4: Performance objectives supplement. Retrieved September 9, 2005, from http://webct.ucf.edu/eme6613c/Unit04/u04info.html

Kubiszyn, T., & Borich, G. (2003). Educational testing and Measurement: Classroom application and practice (7th ed.). Hoboken, NJ: Wiley / Jossey-Bass Education.

Reigeluth, C. (1999). Instructional-design theories and models: 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates.

Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2006). Teaching and learning at a distance: Foundations of distance education (3rd ed.). Upper Saddle River, NJ: Prentice Hall, Inc.

Stolovitch, H. D., & Keeps, E. J. (eds.). (1999). Handbook of human performance technology: Improving individual and organizational performance worldwide (2nd ed.). San Francisco, CA: Jossey-Bass / Pfeiffer

My writing process

When I start writing a paper for a school assignment, I review each of the requirements of the assignment, making headings for each in a word processor. Then I try to link each part of the assignment with one or more of the objectives for the instructional unit. For example, in preparation for this assignment, I started with three lines at the top of a blank document in the following manner:

What is my writing process like? What is your first step in completing a paper? What is each subsequent step?

Of the four objectives listed for this instructional unit, “recognize and articulate your own writing process”, was the only objective to match the components of the assignment. I try to make the link between assignment requirements and objectives to double-check my interpretation of the assignment.

What I do next, in writing a paper, depends on the type of assignment. In the case of this assignment, I am reflecting rather than summarizing results of research, so I started writing immediately after I linked the assignment requirements with an objective. After each paragraph, I read back through what I said in the previous two paragraphs and make revisions.

Had this assignment been to write about the results of research, I would have started by outlining the sections of the paper. Each section would contain a notation of the applicable requirements in the assignment. If any ideas come to mind for things to write about, I add them as short, incomplete notes in the section it belongs. I create a title page with a temporary title. Next, I start a literature review. The literature review almost always prompts me to change the title of my paper more than once. I only start writing the paper for an assignment when I have a solid topic statement or abstract for guiding the rest of the report.

Whether it be subject of the paper is reflection, research, or a topic where I am a subject matter expert, I revise the report in two paragraph chunks as I go. I think it helps confirm the flow between paragraphs. When I finish a complete section, I skim through the section before moving to the next to check flow. At the end of the paper, I walk away from the computer; final revisions are for a fresh sitting. In the ideal situation, I print the paper to edit with a pencil.

In my first critical revision, I check for using the same word to start sentences and grammar. I only read one sentence at a time to see if it makes sense. When I finish the pencil revisions, I make the corrections in the word processor and make my final read-through on the computer screen for the flow of the paper. Sometimes I move large chunks of paragraphs to other parts of the paper. At this point, I coax my wife into using a red pen on a print-out of the paper. For the most part, she revises for content, leaving comments on what I should leave out, expand on, or add. In ideal situations, I give the paper a third read before turning it in. Unfortunately, procrastination often prevents me from having the energy to put into a third reading.

Technical editing defined

Technical editing is an iterative team effort involving one or more editors, authors, subject matter experts (SMEs), and evaluators. Each team component has specialized roles. The editor provides direction, scope, vision, purpose. Editors verify adherence to applicable style guides as well as review drafts for usability, logic, arrangement, spelling, punctuation, grammar, and design. Writers translate information from subject matter experts to written form. Writers are responsible for arbitrating content presentation between the editors’ global scope and the SME’s specific content. Changes to text during the review process are the responsibility of the writers. SMEs provide expert input on required content. SMEs also serve as a specialist member of the evaluation team to review text for currency, accuracy, and adequacy. Evaluators serve as a third party to improve the effectiveness of delivery. Evaluation is performed in a one-to-one or small group format to include clarity, impact, feasibility, and generalizations. Field trial is performed with evaluators to judge the text’s effectiveness under the targeted conditions.

The editing process is a kind of life cycle. I developed a waterfall style diagram of the production process, consisting of 6 major parts, as shown in Figure 1: define project parameters, logical design, physical design, development, implementation, and maintenance.

Figure 1: Technical Editing Waterfall

Between each step, a review is performed by the editor, SME, and/or evaluators, to be implemented by the writers. Since costs to significantly revise a document increase as the document reaches completion, the review and changes are essential before moving on to the next step.

Alignment between learning objectives and item types in assessments

Method	Type of Test Item	Sample Behaviors
Conventional	True/False	identify, recognize, select, choose
	Matching	identify, recognize, select, discriminate, locate
	Fill-in-the-Blank	construct, develop, discuss, define, generate, identify, solve, locate, state
	Multiple Choice	choose, discriminate, identify, recognize, select, solve, locate
	Short Answer	define, discriminate, evaluate, identify, locate, state, name, select, judge, solve
	Essay	construct, discuss, develop, evaluate, generate, judge, solve
Performance	Performance/Product Checklists	construct, solve, develop, generate, locate, perform, operate
	Assessment Rubric	choose, construct, generate, operate, perform

Alignment according to Gagne's taxonomy:

Method	Type of Test Item	Learning Domain
Conventional	True/False	Verbal Information Concepts
	Matching
	Fill-in-the-Blank
	Multiple Choice
Performance	Performance/Product Checklists	Procedures/Rules
	Assessment Rubric	Problem Solving/Cognitive Strategies

Analysis Phase

The difficult part of this project has not been meeting with the service learning partner or getting the information we need, it has been the deciding on what content is good or bad, correct or incorrect for charting in the goal analysis. At the start, everyone except James wanted to jump into reorganizing the Hospice orientation as we thought it should be, however the assignment called for drawing how the system is currently implemented.

As I sketched out the process of HR orientation with Michelle, we created skill boxes and verbs for what the client wanted to accomplish. Note, what the client wants to accomplish is different from what they are currently presenting, so the charting came to a halt as our group members debated four different interpretations of the assignment. Michelle interpreted the assignment to mean it was possible we were supposed to be sketching how Hospice actually does orientation, now. Since my verbs matched more of an ideal implementation of Hospice, it would have involved a total re-think of the goal analysis. Three of us turned out to be correct anyway, but only in parts. In retrospect, it might have been helpful to have just asked our questions earlier instead of debating our interpretations of the assignment requirements.

One of the most useful things I used during the assignments was the Gagne's list of verbs. I had trouble with adding verbs to the goal analysis because I imagined the verbs had to be some physical action I could observe. I hadn't considered the verb could just as easily been someone speaking a one-word answer to a question. As I started connecting verbal arrows to other boxes in the Visio file I created, I realized I have only a vague understanding of how to create the flow of connections for verbal information. I will need to re-read that section of Dick, Carey, and Carey to properly contribute my input to the team version of the analysis.

In a few partial group discussions, we played with alternative ideas methods for presenting the orientation content in the current program. For example, I think all of my group members believe most, if not all of the paperwork signed during orientation can be done individually, without consuming HR's staff time. By downloading the forms online, receiving them by mail, or coming in to review the current packet on their own shouldn't require as significant amount of time as HR is putting in now. If HR allowed employees alternative methods or opportunities to file personnel paperwork, they could use more orientation time to orient the new employees to the culture of the client's organization.

After reading draft assignments from other students in class, I had a new appreciation for following the flow of charts. Creating the proper breaks in flow for pagination, drawing arrows in the right direction, and picking the best verbs makes a big difference in what information a chart conveys. Even though 2c said the goal statement in the draft I presented in class was acceptable, I believe after re-reading Unit 1 Supplement, it can be improved to better clarify the learner, the real-life context, and the available tools for accomplishing goals. Right now, the goal I stated feels very heavy in what learners will be able to do. Michelle and James far exceeded my expectations of time commitments to this group project, and I have met with them in person and by phone on several occasions outside regularly scheduled face-to-face class meetings. I will be meeting again with James on Monday to clarify terms in the team learner and context analysis.

The attached assignment includes a goal and subskills analysis, and learner and context analysis of a non-profit.

Attachment	Size
HOTC Learner&Context Analysis 20051002 blacked out.pdf	64.56 KB
HOTC goal&ssa 20051002 blacked out.pdf	56.35 KB
HOTC analysis phase feedback.pdf	45.45 KB

Bloom's taxonomy

Measurable Action Verbs for Generating Performance Objectives based on Bloom’s Taxonomy for Learned Outcomes.

Knowledge Level	Comprehension Level	Application Level
Arrange Acquire Define Distinguish Duplicate Identify Label List Match Memorize Name Order Recognize Recall Repeat Reproduce	Classify Demonstrate Describe Discuss Distinguish Explain Express Identify Interpret Locate Recognize Rephrase Report Represent Restate Review Select Sort Tell Translate Transform	Apply Classify Demonstrate Dramatize Develop Employ Generalize Illustrate Interpret Operate Organize Prepare Practice Reconstruct Relate Schedule Sketch Solve Transfer Use
Analysis Level	Synthesis Level	Evaluation Level
Analyze Appraise Calculate Categorize Classify Compare Contrast Criticize Deduce Detect Diagram Differentiate Discriminate Distinguish Examine Experiment Inventory Question Test	Arrange Assemble Collect Compose Construct Create Derive Design Document Formulate Manage Modify Organize Plan Prepare Produce Relate Set up Synthesize Tell Transmit Write	Agree Appraise Argue Assess Assume Attack Challenge Choose Compare Contrast Decide Defend Estimate Evaluate Judge Predict Rate Score Select Support Validate Value

Chunking instructional units

Alternative Methods for Chunking Units into Lessons based on 5E Instructional Strategy

Instructional Unit 1
Instructional Event (Engage)	Lesson 1
Instructional Event (Explore)
Instructional Event (Explain)
Instructional Event (Elaborate)
Instructional Event (Evaluate)
Instructional Unit 2
Instructional Event (Engage)	Lesson 2.1
Instructional Event (Explore)	Lesson 2.2
Instructional Event (Explain)	Lesson 2.3
Instructional Event (Elaborate)	Lesson 2.4
Instructional Event (Evaluate)	Lesson 2.5
Instructional Unit 3
Instructional Event (Engage)	Lesson 3.1
Instructional Event (Explore)	Lesson 3.2
Instructional Event (Explain)
Instructional Event (Elaborate)
Instructional Event (Evaluate)	Lesson 3.3
Instructional Unit 4
Instructional Event (Engage)	Lesson 4.1
Instructional Event (Explore)
Instructional Event (Explain)	Lesson 4.2		Lesson 4.3
Instructional Event (Elaborate)
Instructional Event (Evaluate)	Lesson 4.4
Instructional Unit 5
Instructional Event (Engage)	Lesson 5.1		Lesson 5.2
Instructional Event (Explore)
Instructional Event (Explain)
Instructional Event (Elaborate)
Instructional Event (Evaluate)
Instructional Unit 6
Instructional Event (Engage)	Lesson 6.1
Instructional Event (Explore)
Instructional Event (Explain)	Lesson 6.2	Lesson 6.2		Lesson 6.4
Instructional Event (Elaborate)
Instructional Event (Evaluate)
Instructional Unit 7
Instructional Event (Engage)	Unit Overview
Instructional Event (Explore)	Lesson 7.1
Instructional Event (Explain)	Lesson 7.2
Instructional Event (Elaborate)	Lesson 7.3
Instructional Event (Evaluate)	Unit Summary
Instructional Unit 8
Instructional Event (Engage)	Unit Overview
Instructional Event (Explore)	Lesson 8.1	Lesson 8.2		Lesson 8.3
Instructional Event (Explain)
Instructional Event (Elaborate)
Instructional Event (Evaluate)	Unit Summary

Defining performance objectives

Dick, Carey, and Carey (2005) outline a set of requirements for performance objectives in an instructional unit, including audience, behavior, condition, and degree. The audience is the target population to complete the assigned objective. Behavior includes an action verb to describe what the learners are expected to know and/or be able to do. Cues, stimuli, tools, scope, and complexity are part of the condition in a performance objective. Finally, the acceptable performance is defined by the degree. A complete objective would have all four components, for example, "provided with a list of insects, first graders will write the list of insects in alphabetical order with 90% accuracy."

Gagne verbs

Domain	Measurable Verbs/Behaviors
Verbal Information Names, labels, facts or a collection of propositions	Arrange Acquire Define Distinguish Duplicate Identify	Label List Match Memorize Name Order	Recognize Recall Repeat Reproduce
Concepts A set of objects, symbols or events grouped on basis of shared characteristics that can be referenced by a name or symbol.	Classify Describe Discuss Distinguish Explain Express Report	Represent Identify Interpret Locate Recognize Rephrase Restate	Review Select Sort Tell Translate Transform
Procedures & Rules Relational rules or principals and procedural rules or procedures.	Apply Classify Demonstrate Dramatize Develop Employ Generalize Illustrate	Interpret Inventory Operate Organize Prepare Reconstruct Relate Practice	Set up Schedule Sketch Solve Transfer Use
Problem Solving Combine learned principles, procedures, verbal information and cognitive strategies in a unique way within a domain to solve original problems	Analyze Appraise Assemble Calculate Contrast Criticize Compose Construct Create	Deduce Detect Derive Diagram Design Estimate Examine Experiment Formulate	Hypothesize Plan Produce Question Synthesize Test Validate
Cognitive Strategy Internally organized skills whose function is to regulate and monitor the utilization of concepts and rules	Agree Appraise Argue Assess Assume Attack Challenge	Choose Compare Contrast Decide Defend Evaluate Judge	Modify Predict Rate Select Support Synthesize Value

Instructional Treatment Plan

The instructional treatment plan builds on the analysis phase of the instructional design process. Some of the elements of the subskills analysis could be plugged right into the assessment alignment table. The organization I've been working with has an uncommon element of attitudinal and emotional importance, therefore the objectives of this plan include attitudinal elements otherwise not seen in a "proper" instructional treatment plan. Version 2.1 is a draft and the name of the organization has been removed since I didn't ask them about posting this to the internet.

One of the major stumbling blocks in completing this assignment was choosing a grounded instructional strategy. The assignment had a template, however the default Gagné instructional strategy seemed to be better suited for classroom, face-to-face instruction. Part of the effort in this project has been to present a large part of the orientation material in a web-based form, and WebQuests, paired with job aids, seemed like the closest grounded instructional strategy to our goal.

In the process of researching WebQuests, I developed a list of helpful resources such as the WebQuest template. I even signed up for a QuestGarden account, a PHP-based site for generating WebQuests. My professor, Dr. Hirumi, studied under faculty at SDSU. A guide on the building blocks of a WebQuest was also helpful in guiding the instructional strategy details for the treatment plan in my assignment.

During this assignment, I realized I'm rusty on rules for creating surveys. I used to know some good tips on trying to dumb-proof surveys, but I need a refresher. I think the Ph.D. program will have a survey course later in my plan of study. During the implementation of assessment tools later in this project, I may need to refer to my old marketing research textbook for some of those rules on survey creation. All I remembered to do was to not make the feelings about HR a double-barreled question.

After receiving feedback from Dr. Hirumi about version 2.2 of the instructional treatment plan, it was easier to see just how much the WebQuest instructional strategy was being modified to fit the objectives of the instructional unit. Tracy gave an example of a WebQuest she had used before. The task was to research George Washington. The process was to visit a website, read about George, answer some questions, and write an essay. The resources were the initial links provided to the learner, and the evaluation was over the essay. In the case of the client for this project, much of the "questing" is just browsing documentation prepared specifically to guide the learner through filling out their new hire paperwork.

As a result of the feedback from Dr. Hirumi, each part of the plan received some sort of significant modification. Initially, the plan included content to teach some attitudinal content, but that was removed in the final version, because we thought it would be better to allow hospice culture subject matter experts to develop a face-to-face presentation on what they wanted new employees to know. The initial objectives were choppy, and didn't follow 2c's ABCD rule for objectives. The instructional plan also lacked some sort of activity to reinforce the information in the readings. What I imagine for the Accident/Incident Report lesson is to present the new employee with three scenarios. They will need to identify which one requires an Accident/Incident report, then take information from the scenario to fill out a practice form.

The important thing to monitor as revisions were made to the plan is continuity from one part to another. At one point during the revision process, the objective for the learning about the employee assistance program only required learners to identify uses of the program, when in fact the learner also needed to know it was free, voluntary, and confidential.

The other continuity problem surfaced during revisions when we added scenario activities to the process. The objectives only accounted for making choices based on the Hepatitis B information sheet. Since the learners should know from the objectives they will have to apply information from a scenario in addition to the Hepatitis B information page, it was necessary to go back and amend the objectives. There was some discussion during the revision that adding all that information to the objectives "isn't how things are normally done". That was the problem though, the way people normally write objectives is incomplete and/or doesn't align the instruction and evaluation. At this point, the one bit of knowledge of aligning objectives is a reoccurring issue that will probably be one of the educational highlights of this Ph.D. program.

Attachment	Size
EME6613_HOTC_Instructional_Treatment_Plan_2.2_blacked_out.html	14.64 KB
EME6613_Instructional_Plan_Final_Feedback.doc	90.5 KB
EME6613_Instructional_Plan_Preliminary_Feedback.doc	76 KB

Assignment 1 - Autobiography and Expectations

I grew up between Galveston and Houston, Texas. I graduated from The University of Texas at Tyler with my business degree and moved to Orlando for UCF. I work for a company based in Tyler, TX as their technology director. I help facilitate training to small, paper-based insurance agencies to help automate and streamline their business practices.

While I was at UT, I took a course about marketing in China. As part of the course, we spent time in Beijing. In addition to visiting popular tourist attractions, we experienced some of the business aspects of China from the US Department of Commerce office and a small law and accounting firm. From the US Department of Commerce office, we got a view of their experiences working between US businesses and the Chinese government. What I found surprising was it was easy to set up shop in China, but difficult to maintain stability. Often the role of the commerce office was to mediate, usually unsuccessfully, issues where the Chinese government nationalizes US business operations. Copyright and patent issues were also a problem I was educated about. Chinese enforcement of US patent law is not common.

The law firm we learned from in China was primarily engaged in trying to resolve issues with the accounting practices of businesses in China. Bribes and mismanagement of funds within companies were commonplace for their clients. A large percentage of businesses became insolvent, not because the books were disorganized or "cooked", but just because gatekeepers were bribed to spend, take (or not take) actions, or otherwise inappropriately direct the company they worked for.

More important I think was the cultural immersion experiences. For example, my wife and I visited a McDonald's to compare the menu and food with the USA. While the menu, kitchen, decor, and uniforms were all the same, each employee took more pride in their work. An employee greeted us when we got our food just to make sure we were able to find a seat in the dining area. The buns were fluffier, the patties were centered in the bun, the wrapper was carefully folded around the hamburger, and the condiments seemed to be organized on the sandwich. The care they put into making our food was so extraordinary that we even noticed it for fast food.

Beijing is a heavily populated area. The air is heavily polluted, parts of the town are complete shambles, and not everyone has a car. It wasn't until we returned from China where we realized from rich to poor, the Chinese we encountered had very little sense of entitlements. People who lived in one room apartments with a single light in the ceiling, a bed, and plumbing were happy to have a place to live. It was a stark contrast to what I would see on CSPAN about the new Medicare plans that were being debated when I returned.

My interesting fact is my car. When I was in high school, my first car was a minivan by choice and has been my recurring car style purchase since.

My definition of culture is the sense of purpose a group of people have in life. Stereotypes of USA culture say people should grow up, get educated, get married, buy a house, have kids, and retire to a life of cruise ships and RVing. The culture to participate in the culture, the things you do in life should contribute to the success of that family and the business you work for. I believe there are four basic elements to create culture differences: geographic distribution, religion, language, and history.

My knowledge of cross cultural training comes from the orientation I had before flying to China and from leading a software development project with members from European countries. Language seems to be the biggest barrier I've experienced. It can lead to misunderstandings that were never meant. I've heard others cite cultures where something like waving at someone inappropriately or speaking at a time they shouldn't, can be offensive, however I haven't experienced that. The only example I can provide is an Indian family I knew when I grew up that would always point with their middle fingers. They just thought they were pointing and never understood they were giving the bird to everyone who saw.

My perception is the internet has caused not only training, but a large percent of global communication in general to move to English formats. If I could pick one international trend, I would say learning English is the biggest international trend right now.

My goal in this course it to take something worthwhile back to the company I work for that will help them with training customers the company is starting to acquire in Central and South America.

Reflective paper

In my self assessment, I said I wanted to be able to take something back to the company I work for to help with training new customers in Central and South America. I got perspective on the troubles a foreign expatriate might experience when visiting the USA. What I will be able to bring to the company I work for will actually be related to hiring people from overseas. I will be able to prepare a set of tips for hiring foreigners since training the existing employees how they could improve their interactions with a new employee, will help the new employee adapt and be more successful.

<!--pagebreak-->

A recurring training practice I noticed was a self-awareness goal. One thing I expected to see a lot of in this course was content on specific differences between major cultures. None of the research or our class discussion seemed to emphasize reading pamphlets of differences between cultures. I can recommend when companies send employees overseas they should send people with the best ability to adapt to a different environment, not necessarily the person with the highest technical expertise, as is the common practice. As I said in assignment 2, by getting training participants to know they will have to make an effort to observe, internalize, and adapt to new host cultures, as expatriates, they will be much better off. Experiential learning methods seemed to be the best way to create adaptive expatriates

<!--pagebreak-->

I know using surveys or assessments to screen expatriate candidates can improve success in the host environment. An obvious training tip I hadn't thought of before this class is the importance of language training for expatriates planning on making a long-term commitment. The class discussion about the importance of educators in Asia was enlightening. I would have liked to have tried out a cross-cultural training workshop like I found in Delany-Barmann and Minner's 1996 publication from assignment 2 without knowing what the workshop was beforehand.

<!--pagebreak-->

This was the first class I actually had to use the university's online, library, journal, search system. It didn't help me with using a search engine so much as it just got me familiar with resources I haven't used before. I also realized during this semester the type of assignments I created for this class can be part of an online portfolio of research. After the end of last semester, the files I posted to my WebCT site were deleted. By putting the files on my website, I can keep them as a reference for myself and for Google searches. I don't think I'm the only person that has finished a class only later to realize 3 months later I did a brain wash of 70% of the things I learned earlier.

<!--pagebreak-->

The biggest part of the course I will take away isn't directly related to cross-cultural training. I will be able to apply the class discussion regarding the evaluation of training design with proper linking of objectives to assessment.

Doing the research to find information on cross-cultural training was informative, however I would have liked to had more professor-recommended cross-cultural training advice. Instead of assignment 5, I would have rather had a face-to-face meeting for professor-recommended training. I didn't feel like I got a lot out of assignment 5 or the class discussion.

<!--pagebreak-->

Each assignment was time consuming enough to take place of a class meeting to justify the mixed-mode design of the course. I can't speak for everyone, but it took me, give or take, 6 hours to complete each assignment from the time I started researching to when I posted the report. I'm sure I absorbed some information from all the publications I wasn't able to use as references. If the assignments were easier, I would say there should be a face-to-face meeting instead of skipping a class. I would expect if I took the class again to have to do similar research and report writing.

Next time, have the class meet on days where we wouldn't have to miss for Memorial Day. I missed the class time.

<!--pagebreak-->

Questions posed during class discussion to use where commonly, "did anyone else find something like that," but think I the most active discussion during class was when there was disagreement. I think tossing more hypothetical and devils's advocate type comments into the discussion would get more people to freely reference their research to defend their thoughts.

Summary of cross cultural training elements papers from other students

The cross-cultural training programs have a common goal, to assist in the acculturation of their clientele; however, the mediums and their individual focuses differ between the training programs. Many programs use a combination of different mediums to effectively train their clients while others focus on a specific medium. Some programs even use mixed modes in order to provide both a classroom setting and online training modules. In addition some programs instead of focusing on the acculturation will focus on the practicalities of the different cultures in order to provide information necessary for survival in society.

Important aspects common to all training programs include pre-training assessments of language and culture, relocation counseling and face to face instruction. Worldwork, Vital International, and VIP all use modes such a role-playing, simulations, group discussions, and games in order to provide their clientele with several different methods which individuals can learn. In comparison, Cg Associates concentrates on seminars and Window of the World uses a mixed mode of online modules and face to face interactions. Some programs focus on the practicality instead of the actual acculturation. This can be found especially in the University setting where the focus is on financial necessities and information pertinent to survival such as health care, employment, and transportation. Universities instead of providing seminars, or actual training sessions provide a guideline booklet to assist in international students arrivals to the new country. Worldwork and Cg Associates differed drastically in their instruction method between active workshops for Worldwork and passive lectures for Cg Associates. Likely, the optimal solution is to reinforce passive lectures with some sort of hands-on demonstration of knowledge.

Cross-cultural training programs also differ in their evaluation of their clientele. Olive Garden sends employees to train in Tuscany and upon completion trainees are required to demonstrate their training by preparing a meal. This is similar to the Executive Training Program in Japan where trainees must demonstrate their knowledge of Japanese language and culture by working in Japan. The Institute for Medical Education uses testing to determine if their clients have acculturated. Multi Lingua uses language tests to evaluate where students need help, then they offer on-the-spot assistance with phrase translation. One of the assignments predicted increased use of the internet for training and instruction.

Argument for testing in schools

As a graduate of the Texas public education system, I was required, starting in 3rd grade, to pass the Texas Assessment of Academic Skills (TAAS) test for grade advancement and Texas Academic Skills Program (TASP) for entry into The University of Texas for my bachelors degree. I have been in the midst of students who did not pass one or both exams. Opponents of school testing cite research showing long-term damage to students who are retained. In place of standardized tests, strong support systems, high expectations, caring, and more long-term, concept mastery evaluations are suggested (White, 2005, ¶9). I thought in grade school, and continue to think now, the high-profile exams I was required to pass were appropriate queries of my knowledge.

Thesis

Criterion-referenced tests are essential to create credibility and value for diplomas.

Proof

High-stakes tests impact many important decisions including grade promotion, high school graduation, administrative incentives and penalties, and teacher placement (Kubiszyn & Borich, 2003, p. 21). An increasing number of states are requiring students to pass a test to graduate high school, without necessarily providing remediation for students who fail (Kubiszyn & Borich, 2003, p. 21).

Texas has established the Texas Essential Knowledge and Skills (TEKS) academic standards for reading, math, writing, science, and social studies (Student Assessment, Texas Education Agency). In the past, Texas administered the TAAS, and now administers the TAKS test on a criterion-referenced basis. Students and parents receive a report outlining the strengths and weaknesses of the student and a "pass", "pass with academic recognition", or "fail" result for each subject area. Students, parents, and teachers are able to better pinpoint where students and teachers need to concentrate. Over an eight year period, Texas was able to demonstrate, through the TAAS test, an increase in the percent of students meeting minimum state, academic expectations in reading, mathematics, and writing (Texas Assessment of Academic Skills, 2002).

Teachers should be encouraged to break from their bubble of subject-area focus they have traditionally been stuck in from teaching models of the industrial age (Marshak, 2003). A case study in co-teaching revealed how teachers were influenced in states with high-stakes tests and felt compelled to teach only test-related material (Mastropieri, M. A., Scruggs, T. E., Graetz, J., Norland, J., Gardizi, W., & McDuffie, K., 2005). Instead of leaving out non-test teaching materials, teachers communicated with each other to compound educational value in classroom activities, in this case, computer class. In addition to teaching computer modeling, the computer teacher was able to create an activity that used computer modeling to teach world history. Students' test scores improved as a result and students requested copies of the software to take home for additional practice with other subject areas.

Refutation

Opponents of high-stakes testing work on the assumption where high standards are meant to drive low-performing disadvantaged students to work harder. The assumption of the high standards argument is students are all qualified to obtain high school diplomas. While an ideal education system would cycle all students through as educated graduates, not all students have the mental capacity, the personal motivation and persistence, or willingness to fulfill the requirements set forth by the education system. Deficient students are given the opportunity to re-take high-stake tests and should not graduate simply because they showed improvement, a capacity to learn, a willingness to participate, or the persistence to re-take tests and fail. The high school diploma should represent the ability to read, write, and do math for students who can demonstrate their mastery of state academic standards.

An Education Week survey showed "66 percent of teachers thought state tests were forcing them to concentrate too much on what was tested to the detriment of other important topics, and nearly half said they spent a 'great deal of time' helping students prepare for tests" (Doherty, 2002, ¶7). While students should learn more than what is on state tests, it is important for students to master essential, basic subject areas. The requirements on state exams should indeed supersede the education interests of individual teachers.

The American Educational Research Association made a statement concerning high-stakes testing, suggesting accommodations should be made for students not proficient in English (Kubiszyn & Borich, 2003, p. 21). In a country where government, business, and educational transactions are primarily English, high-stakes tests double as an assessment not only of content mastery, but communication skills. The added communication assessment is important, not something to protest. Surveys and research conducted by the government of Manitoba Canada, The Wall Street Journal, and the Association of Legal Administrators found communication and inter-personal skills among the highest demand skills of job applicants (Johnstone & Williams, 2003; Perry, 2002; The Association of Legal Administrators Competency-Based Education Job/Needs Analysis, 2004). Enabling students to avoid gaining proficiency in English does not benefit industries with shortages of job applicants nor the applicants in those industries. Knowing algebra is worthless if you don't understand the question where the details of the algebraic problem is explained.

Conclusion

Criterion-referenced testing is appropriate for schools when linked to the state's academic standards. Teachers can collaborate with others to compound educational activities to make up for time where education is spent directed on passing high-stakes tests. Texas has proven when test content is linked to state academic objectives, teachers and students are able to work together to improve education.

References

The Association of Legal Administrators Competency-Based Education Job/Needs Analysis. (2004, June 28). Association of Legal Administrators. Retrieved on June 27, 2005, from http://www.alanet.org/education/knowledgesurvey.html

Doherty, K. M. (2002, February 27). Assessment. Education Week on the Web. Retrieved June 27, 2005, from http://www.edweek.org/rc/issues/assessment/

Johnstone, P., & Williams, A. (2003, June 19). Manitoba Employer Survey 2000. Government of Canada. Retrieved on June 27, 2005, from http://www.hrsdc.gc.ca/en/mb/survey/employer-shortage.shtml

Kubiszyn, T., & Borich, G. (2003). Educational Testing and Measurement: Classroom Application and Practice (7th ed.). Hoboken, NJ: John Wiley & Sons, Inc.

Marshak, D. (2003, Nov). No Child Left Behind: A Foolish Race Into the Past. Phi Delta Kappan, 8(3), 229-231.

Mastropieri, M. A., Scruggs, T. E., Graetz, J., Norland, J., Gardizi, W., & McDuffie, K. (2005, May). Case Studies in Co-Teaching in Content Areas: Successes, Failures, and Challenges. Intervention in School and Clinc, 40(5), 260-270.

Perry, D. (2002, May 20). Do You Have the Skills Most in Demand Today? The Wall Street Journal. Retrieved on June 27, 2005, from http://www.careerjournal.com/columnists/perspective/20020520-fmp.html

Student Assessment. Texas Education Agency. Retrieved on June 28, 2005, from http://www.tea.state.tx.us/student.assessment/

Texas Assessment of Academic Skills. (2002). Texas Education Agency. Retrieved on June 28, 2005, from http://www.tea.state.tx.us/student.assessment/reporting/results/swresults/august/g310nse_au.pdf

White, J. (2005, June 30). Activity 1-Arguement Against Testing [Msg 1]. Message posted to http://webct.ucf.edu/

Different types of reliability

Type of reliability	When	How	What
Internal consitency	Assess a single dimension	Correlate each individual item score with the total score.	All the items on your test assess the same construct.
Interrater	Find consistency in the rating of some outcome	Examine the percentage of agreement between raters.	The reliability coefficient for your test indicates a poor, moderate, or high degree of agreement between respondents.
Parallel	Compare several different forms of a test to see if they are equivalent or reliable	Correlate the scores from one form of the test with scores from a second form of the same test with the same content.	Two forms of your test are equivalent to one other.
Test-retest	Reliability over time	Correlate the scores from time 1 with the scores of time 2.	The test gives the same results even if the participants didn't all take it at the same time.

Portfolios

A good portfolio is both summative and formative in nature. The contributions to the portfolio should be evaluated as the portfolio is being created as well as a final evaluation as part of a continuous process.

Portfolios should reflect the immediate assignment task and the overall area of study. The content should reflect the interests, in addition to the abilities, of the student.

Sample essay question

Cognitive Objective:

The student should be able to:

• Identify and describe common characteristics of programming frameworks
• Identify and describe issues related to standardized and open source frameworks in a clear and concise manner
• Reference and provide a short description of at least two popular PHP frameworks
• Evaluate the quality of named frameworks

Test Item:

In an article, titled "Why PHP sucks", on bitstorm.org, Edwin Martin complained about not having a standardized framework for PHP. He cited Struts for JSP developers and .Net for ASP developers as models for a standardized framework for PHP. As opposed to a standardized framework, PHP has a multitude of open source frameworks including Midgard, Horde, Blueshoes, Cake, Seagull, Sourdough, binarycloud, SMART, and many others. Compare and contrast having a standardized framework verses multiple privately developed frameworks. Use your analysis to build an argument defending or refuting Martin's complaint. Limit your response to two pages.

Score Scheme:

Points	Score Basis
10	Identified and described common characteristics of programming frameworks
10	Identified and described issues relating to standardized and open source frameworks in a clear and concise manner
10	Referenced and provided a short description of at least two popular PHP frameworks
15	Effectively evaluated the quality of named frameworks
10	Essay follows APA style guidelines, contains no spelling or grammatical errors, and is completed on time.
55	Total Points

Sample test blueprint

Test Blueprint	Categories
	Knowledge	Comprehension	Application	Analysis	Synthesis	Evaluation	Total	(percentage)
Content Outline	(number of items)
1. The student can state the purposes for various string, database, and graphics functions in PHP. (2, 3, 5, 6, 8, 9)	6						6	30
2. Given a line of code, the student will be able to identify parse errors. (1, 4, 7, 10)				4			4	20
3. The student can distinguish standard from PECL modules. (11, 12)	2						2	10
4. Given a programmatic situation, the student can apply a PHP function as a solution. (13, 14, 15, 16)			4				4	20
5. Given generic character types for regular expressions, the student can identify the function of each escape sequence. (17, 18, 19, 20)	4						4	20
Total	12	0	4	4	0	0	20
Percentage	60		20	20			100

Directions
Circle the best answer that either correctly completes each statement or correctly answers the questions.

1. Identify the line of code with a parse error.

1. echo ‘Hello World!’;
2. echo ‘Hello “World”!’;
3. echo “Hello World’s!”;
4. echo ‘Hello World’s’;

2. The purpose of strpos() is to

1. find the first position of an array in a string
2. find the first occurrence of a string
3. insert a string in a specific position of a different string
4. delete the first occurrence of a string

3. The purpose of join() is to

1. add two variables together mathematically.
2. join array elements with a string.
3. concatenate to strings.
4. merge two arrays.

4. Identify the line of code with a parse error.

1. if(strcmp($foo, $bar)) {
2. if strcmp($foo, $bar) {
3. if(substr($foo, $bar)) {
4. if(strtr($foo, $bar)) {

5. The purpose of ksort() is to

1. sort a string starting with capital letters
2. sort an array by values
3. sort an array by keys
4. reverse sort an array by keys

6. $mysqli = new mysqli(‘localhost’, ‘root’, ‘password’); will

1. attempt to open a connection to MySQL
2. return boolean TRUE on success
3. produce a parse error
4. return an array of connection information on success

6. The purpose of mysqli_autocommit() is to

1. turn on or off auto-committing database modifications on InnoDB tables
2. turn on or off auto-committing database modifications on MyISAM tables
3. turn on or off auto-committing database modifications on HEAP tables
4. commit uncommitted SQL transactions

7. Identify the line of code with a parse error.

1. $stmt = $mysqli->prepare(“SELECT * FROM foo WHERE bar=?”);
2. $stmt = $mysqli->prepare(‘SELECT * FROM foo WHERE bar=?’);
3. $stmt = mysqli_prepare($link, “SELECT * FROM foo WHERE bar=?”);
4. $stmt = mysqli->prepare(‘SELECT * FROM foo WHERE bar=?’);

8. break 3; will _________ of a current for, foreach, while, do-while, or switch structure

1. end execution for 3 levels
2. pause execution for 2 levels and resume at the third level
3. pause execution for 3 seconds
4. cause a parse error during the execution

9. The require statement

1. requires a specific variable parameter for continued execution
2. includes and evaluates a specific file
3. requires an end user to enter a password
4. will not cause a parse error on failure

10. Identify the line of code with a parse error.

1. $i += $factor;
2. $i *= $factor;
3. $i #= $factor;
4. $i |= $factor;

Directions
Each statement is either true or false. Circle True for each statement that is true and False for each statement that is false.

11. GD2 is a standard extension

1. True
2. False

12. mailparse is a PECL extension

1. True
2. False

Directions.
Write a PHP function answer to each question in the space below each question.

13. What function would most accurately compare two string passwords?
            strcmp()

14. What function would calculate the sha1 hash of a file?
            sha1_file()

15. What function would convert a string into variables?
            str_parse()

16. How would an administrator remove erroneous spaces from the end of a string?
            rtrim()

Directions
Complete each sentence with the missing PCRE character match.

17. \s represents any _________ character/digit
            whitespace

18. \W represents any __________ character/digit
            non-word

19. \d represents any _________ character/digit
            decimal

20. \w represents any __________ character/digit
            word

Definition of a social scientist

On page 6 of Learning in the Field, by Rossman & Rallis, they discuss the difference between basic research and applied research, then they end the paragraph with mentions of "scientists" and "social scientists", but their research contributions go two different ways. How are "social scientists" not "scientists"? Moreover, if they had reversed the definitions of to read as follows, and if I had just been glossing over the section, it sounds reasonable written either way.

Original: "Scientists" may conduct research that leads to direct practical application; "social scientists" may conduct research that contributes to theory.

Flipped: "Scientists" may conduct research that contributes to theory; "social scientists" may conduct research that leads to direct practical application.

In the original definition, the original authors of the Big Bang Theory would be classified as social scientists? To compound the confusion, the earlier parts of the chapter all appear to say research should promote learning and improve a social circumstance, where then I would argue that direct practical application is best to improve those disordered circumstances. Is the whole thing a misplaced tangent in the flow of the chapter or am I just going to extremes?

In-class writing: still life

Begin time: 7:47pm
End time: 7:49pm
Date: Sept 5, 2006
Location: Classroom Building 1, Room 113 at University of Central Florida
Observer: David Kent Norman
Frame: Classroom observation exercise

Time	ON Observer Notes	OC Observer Comments
7:47	Shadow
	Grey background
	Brick wall
	Building in background	Broken with three horizontal lines that are white
	Top hat on shadow figure	Looks like also wearing coat or cloak and is male body build
7:48	Top of building has extra windows	Windows follow a vertical pattern

Solo observation exercise

Assignment Description

Schedule this the week we do not meet on campus. Duration: 15-20 minutes. Decide on a place, a time, and a frame or question to organize your observation. Make sure you provide a map or photo clearly labelled. If you go to observe a speaker or an event and the event is cancelled, either: 1) reschedule your observation or 2) re-frame your frame or question. (Don't write an observation of kids at the zoo after you arrive and find out the zoo is closed!) Feel free to pick a setting that is interesting for you (like the example where the observations were done at a cliff diving location). Please see the rubric which lists other requirements for this assignment which we looked at in class, like the use of a formal heading. Please use OC, ON, and MN which were covered in the class PowerPoint (See the PPT for an example). Please see the samples, but note both the strengths and the weaknesses evident in these samples.

• Keep in mind the prior work needed to set up the observation
• Practice OC ON and MN beforehand if that helps you
• Print our a grid with columns already labeled if that helps you
• Observe for 15-20 minutes
• Keep in mind you may feel awkward writing
• Consider bringing a book or something to write on & shield your writing

Paired observation exercise

Instructions

Schedule this with a partner this week or the next week (over two weeks) Duration: 15-20 minutes Use: OC, ON, and MN (TN and Spradley’s Matrix, optional) Conduct an observation with a partner Keep fieldnotes, write up your notes into a memo. Exchange your notes and memo via CourseMail. Write a half page to a page responding to your partner’s work (strengths and weaknesses) Submit the Fieldnotes, Memo, and Response from your partner to the HW Box in one document. (Make sure you clearly label the Response with your name and your partner’s name.) Decide on a place, a time, and a common frame or question to organize your observation. Make sure you orient yourselves the same way (for example, label people and the objects in the setting the same way, so that it is easier for you to compare your observations). As with your solo observation, if you go to observe a speaker or an event and the event is cancelled, either: 1) reschedule your observation or 2) reframe your frame or question. (Don’t write an observation of penguins on a glacier after you arrive and find out the glacier has melted!) Feel free to pick a setting that is interesting for you both (like the example where the observations were done in a casino).

• Keep in mind the prior work needed to set up the observation
• Practice OC ON and MN beforehand if that helps you
• Print out a grid with columns already labeled if that helps you
• Observe for 15-20 minutes
• Do not talk to each other once you begin your observation
• You may want to sit apart
• Keep in mind you may feel awkward writing
• Consider bringing a book or something to write on and shield your writing

Article Review: Write-up

The study lead by Judith Correson Caruso (2004) was started as a result of a two part publication by Marc Prensky (2001a, 2001b) where he coined the terms "Digital Native" and "Digital Immigrant" to describe students' use of technology.

Methodology

• Literature review
• Review of the 2003 study
• Quantitative portion of the study included 9350 freshmen and 9050 seniors with 4373 respondents over 13 higher education institutions
• Qualitative portion had 132 students in focus groups over six institutions

Findings

• Technology ownership
• 93.4% of respondents own a computer
• 70.7% and 57.1% of seniors and freshmen respectively owned a desktop
• 38.5% and 52.7% of seniors and freshmen respectively owned a laptop
• 11.9% overall owned PDAs
• Internet access
• 82.2% of on campus freshmen had Internet access through their university
• 56.4% of seniors used a commercial Internet service provider
• Students' satisfaction with Internet service was based partially on the institutions IT environment
• Students are often frustrated with crowded computer labs
• Patterns of technology use
• The applications students use on their electronic devices are primarily for educational purposes, secondarily for communication, and lastly for presentation
• Students often used computers for writing documents, sending e-mail, and surfing the Internet simultaneously.
• Hours of technology use
• Academic computer use has a strong relationship to students' academic major and class status.
• Level of skill
• Students gave themselves high ratings with regards to the skill they had attained with respect to applications for communication, word processing, and the Internet.
• Seniors ranked themselves higher with regards to tools such as Powerpoint and spreadsheets.
• Qualitative and quantitative responses differed with regards to the skills students attained. Quantitative results showed students had all the skills they need whereas qualitative results showed students only had enough minimal skills to accomplish specific tasks, with no problem-solving skills or in-depth application knowledge.
• Information technology in the classroom
• Students made a note of preference for technology in the classroom
• Previous positive experiences with technology had a major impact on the preference for classroom technology
• GPA did not have a significant impact on technology preferences
• Impact of technology in the classroom
• The activity with the highest impact was "helped me to better communicate with the instructor".
• Benefits of the use of technology in the classroom
• Students identified convenience as one of the primary benefits of using IT in class.
• Good use of technology helped students learn
• Barriers to the use of information technology in the classroom
• 16.7% of students reported using IT in the classroom felt like extra work
• 14.1% of students reporting having trouble running applications on their computer
• 13.4% of students had limited access to a printer
• 9.7% of students had a lack of technical support
• Course management systems
• Use of technology was most often defined as using a course management system
• Students who had a stronger preference for technology in the classroom had a lower preference for using a course management system
• There was a positive correlation between a higher percentage of students using a course management system with a higher positive assessment of the content management system at that institution.
• Course management system features used
• Students Perceived benefits of using a CMS

Future trends

• Information technology use in the classroom is primarily for convenience rather than for improving learning
• Six trends are likely to lead the way for improved information technology adoption for instruction:
• Data mining
• Increased student and instructor information literacy to pair with academic standards
• Improvement of information technology quality and usability
• Proliferation of scholarly information
• Integration of new technologies, potentially including virtual reality, simulation, video games, or modeling
• Improved planning and creation of work plans for implementations of technology in learning

Conclusion

• Technology is primarily being used for convenience, communication, and management
• Results are only a snapshot
• Students have a working understanding of technology, but need more in-depth improvements

Caruso, J. B. (2004 September). ECAR study of students and information technology, 2004 Convenience, connection, and control. Educause Center for Applied Research. Retrieved on September 5, 2006, from http://www.usit.uio.no/it/suit/stud2004-rapport/ecar.pdf

Prensky, M. (2001, October). Digital natives, digital immigrants. On the Horizon 9(5). Retrieved November 25, 2005, from the Marc Prensky Web site: http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf

Prensky, M. (2001, December). Do they really think differently? On the Horizon 9(6). Retrieved November 25, 2005, from the Marc Presnsky Web site: http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part2.pdf

Theory: Write-up for Conceptual Framework

In a two part publication by Marc Prensky (2001a, 2001b), he outlined changes he believes have been affecting the decline of education in the US. Prensky makes the assertion that the thinking patterns and brains of today's students have fundamentally and irreversibly changed, to a point where our education system is not designed to teach to the new thinking patterns. "Digital Immigrants," those instructors of the pre-digital age, struggle to teach "Digital Natives," who's "native language" is grounded in electronics. Though Prensky cites some indirect evidence to reinforce his claims, he confesses he had not directly observed Digital Natives (2001b). This study concentrates on investigating Prensky's specific claim that Digital Natives prefer their graphics before text (2001a).

His list of causes for change includes computers, video games, digital music players, video cams, cell phones, and other digital toys. He also makes the claim that students are spending six times more of their time playing video games and watching TV than reading. Prensky named his new breed of student the "Digital Native." The remaining population, who was not born into a digital lifestyle, is conversely named "Digital Immigrant."

The evidence Prensky (2001b) cites for why Digital Natives think differently is based in neuroplasticity, social psychology, and studies of children using games for learning. He discusses psychological malleability, attention span, and the design of games as variables within each respective category of evidence. He claims the difference between Natives and Immigrants is evident in teaching styles and is the cause of why Natives can't pay attention to instructors.

Prensky identifies Digital Immigrants as a population more likely to use the Internet as a secondary resource, print documents rather than review them on a computer screen, and call email recipients to confirm receipt of emails. Part of the claim of difference between Natives and Immigrants is a language barrier, much like children might only know about turntables or phonographs from a history class. Digital Natives are accused of having "the attention span of a gnat" for old ways of learning, favoring instead anything else (2001b). As such, Prensky outlines five generalizations for the preferences of Natives and Immigrants which he thinks affect learning.

Prensky (2001a) believes Digital Natives prefer to receive information quickly, parallel process, and multi-task. He contends Natives prefer random access to resources, graphics in presentations before text, do best when they are networked, and have instant gratification, with frequent rewards. Finally, natives prefer games to "serious" work. Conversely, Prensky's Immigrants prefer performing linear tasks slowly, individually, as part of a serious process.

One explanation Prensky offers for the evolution of digital migration relates back to adoption in cultural migration theory. In cultural migration, children easily adopt new ways of the culture, forcibly resisting the old. It is the older members of the population who are forced to adopt the new ways.

Eggen and Kauchak (1999) say a learning experience involves dispositions and attitudes, metacognition, and general transfer of knowledge. In fact, the more widely recognized process of learning is more complex. The Modal Model (Bruning, Schraw, Norby, & Ronning, 2004; Healy & McNamara, 1996) defines information processing in terms of sensory memory, short-term memory, and long-term memory. Sensory and short-term memory is limited by attention, prior knowledge, and the context in which experiences occur (Bruning et.al., 2004).

Attention is important for converting cues in sensory memory to working memory to be processed for long-term memory (Ormrod, 2006). Without the transfer of inputs from sensory memory to the higher parts of the memory chain, learners cannot rehearse information in short-term memory to remember the inputs in long-term memory (Bruning et.al., 2004; Eggen & Kauchak, 1999). Complaints of students not paying attention in the classroom is not new (Kassinove & Summers, 1968; Wetstone & Friedlander, 1974). Gagné (1969) made his attention research the top-most important part of his instructional strategy. Research of graphics in education before the "digital age," shows evidence that students prefer colors in presentations (Gaines, 1970). The Gaines publication references 29 publications related to color-forms acting as a instinctual stimulus or preference for children. The placement of items on the screen determines the importance of the content; items higher on the screen appear to the viewer to be more important and attract attention (Thorsen, 2006). The content at the top of the screen should be used to grab students' attention.

Schema theory relates to reader expectations for inputs (Garner, 1987). When incoming information fits readers' expectations, the information can be encoded into memory quickly (Garner, 1987). In technical prose, comprehension schemas are based on the extraction of information based on extracting the microstructure from text and deriving a macrostructure to serve as the "gist." The macrostructure parts are stored in memory and are used for future memory expansion, recall, and inferences (Kintsch & van Dijk, 1978). Readers' goals are based on their within-culture "textual-schemata", which can be predicted based on what readers consider relevant based on their existing macrostructure (Garner, 1987). Readers' existing schema classifies all propositions of inputs as either relevant or irrelevant (Kintsch & van Dijk, 1978).

Motivation is a key component behind the dispositions and attitudes of learning situations (Mueller, 1992). The field of behavioral psychology, or more simply Ivan Pavolv's conditioned response study, may serve as some explanation for graphical preferences of Digital Natives. Combined with cognitive psychology, or the relationship between environmental events and their outcomes, organisms learn particular situations produce particular results (Mueller, 1992). Digital Natives' experience with favorable results from graphical activities may also have a link in cognitive psychology.

All living organisms must categorize experiences to survive since not every situation should be treated the same (Smith, 2004). Learners also have a general state of prediction and expectation (Smith, 2004). The result is that we are far more likely to care about what is going to happen in the future than what is happening right now. Smith says prediction is the core of reading because it cuts down on the number of possible alternatives when we decide what to do with what our eyes are looking at. One of the constraints of prediction is prior experience and knowledge (Smith, 2004).

Smith goes on to discuss two sides of reading, which he names visual and non-visual. The more non-visual information a person has, the less visual information they need to understand what their eye are seeing and vice versa. When reading is difficult, it is because of a deficit in one of the two areas of visual or non-visual input, the link between the brain and visual input can be a bottleneck and cause functional blindness. The functional blindness causes critical information for understanding to not be passed down the memory chain: sensory memory, short-term memory, long-term memory. This point is only exasperated by the link to prior experience where Klausmeier, Ghatala, and Frayer (1974) found prior experience can lead subjects to ignore parts of a later task. Here, the preference of Prensky's Digital natives to have graphical presentation first is a result of the failure for the learners to analyze text stimuli in sensory memory as a cue fore relevant response (Mueller, 1992). On the other hand, noise in a message, irrelevant details, or lack of relevant prior knowledge may serve as a distraction, disruption, or activation of the wrong prior knowledge in a learning experience (Clark & Lyon, 2004).

To date, there are no theories or conceptual frameworks to link Prensky's observations with a solid foundation of research. The preferences of learners may be impacted as a result of conditioning to specific environments; however, even with differing backgrounds of electronic exposure, digital immigrants and digital natives should have statistically insignificant differences exposed to a similar, base set of optimal learning conditions which fit the learning abilities of students outside the window of digital aptitude. Clark and Lyons (2004) give a formula of conditions to support learning with six psychological events, which may support higher level framework of learning. They also put differences between learners on the same level of importance for creating optimal learning conditions, citing prior knowledge and special ability as limiting variables. While using Doom, a classic computer game, in a lesson to teach 20-year-olds may not have the same impact on Digital Immigrants, the reason Digital Immigrants may not have similar impact is not because of limited digital experience per-say, but rather that they simply don't share the same prior knowledge as the Digital Natives who played Doom. There would be a similar situation within a group of Digital Natives if some had not played Doom when others had; the link to building on prior knowledge would be different.

References

Bruning, R. H., Schraw, G. J., Norby, M. M., & Ronning, R. R. (2004). Cognitive psychology and instruction (4th ed). Upper Saddle River, NJ: Pearson Merrill Prentice Hall.

Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29. Retrieved on July 27, 2006, from the University of Southern Queensland Web site: http://www.usq.edu.au/material/unit/resource/clark/media.htm

Clark, R. C., & Lyons, C. (2004). Graphics for learning: Proven guidelines for planning, designing, and evaluating visuals in training materials. San Francisco, CA: Pfeiffer.

Eggen, P., & Kauchak, D. (1999). Educational psychology: Windows on classrooms (4th ed.). Upper Saddle River, NJ: Merrill Prentice Hall.

Gagne, R. M., & Rohwer, W. D., Jr. (1969). Instructional psychology. Annual Review of Psychology, 20, 381-418.

Gaines, R. (1970 December). Children's selective attention to stimuli: Stage or set? Child Development, 41(4), 979-991.

Garner, R. (1987). Metacognition and reading comprehension. Norwood, NJ: Ablex Publishing Corporation.

Healy, A. F, & McNamara, D. S. (1996). Verbal learning memory: Does the modal model still work? Annual Review of Psychology, 47, 143-172.

Kassinove, H., & Summers, M. (1968 January). The developmental attention test – A preliminary report on an objective test of attention. Journal of Clinical Psychology, 24(1), 76-78.

Kintsch, W., & van Dijk, T. A. (1978). Toward a model of text comprehension and production. Psychological review, 85(5).

Klausmeier, H. J., Chatala, E. S., & Frayer, D. A. (1974). Conceptual learning and development: A cognitive view. New York, NY: Academic Press, Inc.

Mueller, R. J. (1992). Instructional psychology: Principles and practices. Champaign, IL: Stipes Publishing Company.

Ormrod, J. E. (2006). Essentials of educational psychology. Upper Saddle River, NJ: Pearson Merrill Prentice Hall.

Prensky, M. (2001, October). Digital natives, digital immigrants. On the Horizon 9(5). Retrieved November 25, 2005, from the Marc Prensky Web site: http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf

Prensky, M. (2001, December). Do they really think differently? On the Horizon 9(6). Retrieved November 25, 2005, from the Marc Presnsky Web site: http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part2.pdf

Smith, F. (2004). Understanding reading: A psycholinguistic analysis of reading and learning to read (6th ed.). Mahwah, NJ: Lawrence Erlbaum Associates.

Thorsen, C. (2006). TechTactics: Technology for teachers second edition. Boston, MA: Pearson Education, Inc.

Fieldwork: Interview Transcript and Memo

Instructions: Duration 20 minutes. Interview one person (NOT your partner) using your questions for your mini-study and then share the transcript with your partner for peer review. Here you are: 1) practicing the interview process and 2) testing your questions.

Please follow the rubric and submit (as one document) your transcript, the related memo and any field notes (optional). Clearly label each section. Note that your peer review is a separate assignment. Once you have transcribed your interview, please send the transcription and your memo to your peer for critque, and likewise give them written feedback for their work on the interview. Please see Padgett for details on the transcription.

Fieldwork: Peer Review You Received for your Interview

Instructions: Please upload the feedback you received from your peer on your interview transcript here, along with your brief response either: a) responding to the comments or b) documenting your own reactions to reading someone else's interview. (2-3 pages single-spaced total) DO NOT include the transcript or memo from YOUR OWN interview. Please title the document using YOUR last name before you upload as required in the Protocols, but also please make sure the reviewer's name is clearly identified on the top of the document.

Proposal: Presentation Slideshow

A Pilot Study for EDF 7475: Instructional Technology Students' Perceptions of How Age Impacts Proficiency with Electronics

Problem: Marc Prensky makes his conclusions ("research") based his observation from memory
Question: Are Prensky's observations reproducible in the minds of other educators?
Significance: Impacts presentation of content to learners of different backgrounds.

See our texts...

Lit review: Scarce supply of academic research related to Prensky; industry uses his coined terms
Clark & Lyons (2004) "Graphics for Learning" provides a base conceptual framework
Ask educators if their experience matches Prensky

See our texts and the readings related to the IRB

• Population: UCF Instructional Technology listserv
• IT Chair leading research on Prensky
• Researcher has been studying adult learning theory and how varying display formats affects learning

Data

Triangulate:
Distribute web-based questionnaire
Cite similar studies
Quote existing adult learning theory

Interpretivism:
Why?
No validated instrument exists to measure "digital propensity". Even lead researchers in graphics for learning (Clark & Lyons) don't make a definitive declaration to support or refute Prensky.
How?
Use triangulation to compare the results of the questionnaire to educators with existing similar studies and adult learning theory.

Methodological Limitations

Differential selection and selection maturation interaction: Subscribers to the IT listserv have likely had exposure to the topic from professors and have background that may skew their responses.
Self reporting error and bias
Random sampling error: Questionnaire is available to every subscriber. Only particular people may respond.
Instrumentation: Un-piloted questionnaire may not test what it is meant to test

Next steps

• Dissertation topic "An investigation of the relationship between insurance employees' digital propensity and their reactions to instruction"
• Helps form the basis for a hypothesis of the outcome of the dissertation topic

Works Cited

Warne, D. (2006, August 23). Unlock work internet or risk losing staff: Microsoft. APC Magazine. Retrieved on August 23, 2006 from http://www.apcstart.com/site/dwarne/2006/08/1104/unlock-work-internet-or-risk-losing-staff-microsoft

Clark, R. C., & Lyons, C. (2004). Graphics for learning: Proven guidelines for planning, designing, and evaluating visuals in training materials. San Francisco, CA: Pfeiffer.

Prensky, M. (2001, October). Digital natives, digital immigrants. On the Horizon 9(5). Retrieved November 25, 2005, from the Marc Prensky Web site: http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf

Prensky, M. (2001, December). Do they really think differently? On the Horizon 9(6). Retrieved November 25, 2005, from the Marc Presnsky Web site: http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part2.pdf

Mini Study Field Notes/Journal

Date	Entry
9/9/2006	Mom died. Leaving for Houston tonight.
9/12/2006	Finally got some dial-up Internet access. I received an email back from Tracy Dietz asking questions about my IRB application and sent a response today, too. She asked for a copy of the informed consent document, which I thought I sent in the first application, but sent a copy again.
9/13/2006	Apparently I don't understand what the informed consent document is that Tracy Dietz wants, because she said I didn't send it again. I tried to email Dr. Blasi to get her advice on what I should be sending to IRB, but can't stay connected long enough to send the email.
9/17/2006	Back in town. Way behind on everything having to do with regular life. Emailed Dr. Blasi about IRB informed consent problems.
9/24/2006	Quit my job to catch up with school. I haven't done a single assignment for my Wednesday night class all semester. Behind in reading for Tuesday night and need to start doing background research for comprehensive exams.
9/25/2006	Dr. Blasi hasn't replied. Perhaps I should have asked her through WebCT email instead of regular campus mail.
9/29/2006	Dr. Blasi reminds there is no IRB approval letter for my study yet. Perhaps time to try a new approach.
10/1/2006	Read about Zoomerang online survey software and got an account. Got familiar with the interface to build a survey online. I think this will be a perfect tool for getting anonymous surveys online.
10/4/2006	Asked Dr. Hirumi to bump my comprehensive exams back two weeks. Needed more time to do research for mini-pilot and comps. With luck, I'll be able to double up on research between the two projects.
10/5/2006	Discovered the adult learning textbooks from last year might actually be helpful in the research behind Prensky. Bruning, Schraw, Norby, & Ronning planted some seeds about expertise related to Prensky's propositions. Should ask Dr. Hirumi what he thinks.
10/9/2006	I entered the questionnaire into Zoomerang and put the informed consent document on the page before the survey starts. Then I duplicated a large part of the informed consent to the first contact email and mixed it with some of the Tailored Design Method. I sent the Zoomerang output and initial contact to IRB for another try.
10/10/2006	Something I did changed IRB's opinion on my informed consent. Perhaps it was actually seeing the flow of clicking on Zoomerang to know informed consent must be viewed before the survey to make the difference. Barbara Ward is expected to approve the IRB application. Also started on the Powerpoint for the proposal presentation for class. Identified Ruth Clark as one of the most likely to be helpful during the research behind the validity of Marc Presnky's claims.
10/13/2006	Meet with Roy Marquez to brainstorm about Prensky's propositions. Roy found an article by Kalyuga, Chandler, and Sweller that appears to serve as a springboard for finding more relevant articles regarding expertise and instructional presentations. I identified "broadcast" and "interactive" as important key words for discussing Prensky. Roy and I discussed the revisions to Prensky to say 1) people gravitate towards interactivity, 2) Digital Natives choose interactivity over work, 3) Digital Immigrants choose familiarity over interactivity. We posited that interactivity leads to exchange of ideas and information, and stimulus with regards to creating interactivity involved the major senses. Discussed propositions by Don Tapscott to say the Internet forces natives to search for information, therefore it develops their thinking and investigation skills; kids learn things easier; that the n-gen doesn't have to memorize as much because they can type it in word and recover it later; even with a supposed short attention span, n-gens can still spend hours on the computer Posited that "discovery learning" allows students to click on their interests and therefore have the instant rewards Prensky identified; familiarity with regards to immigrants or experts leads to the equivalent of convenience and rituals surrounding those conveniences; comfort with text means learning game skills on a computer is too much effort if they're already skilled with text; experts/immigrants are used to broadcast forms of information and may or may not have a relationship to Pavlov studies of conditioning; developed a "hierarchy of interactivity," thought to be related to why young people might choose computers over reading a book.
10/17/2006	Had another brainstorm meeting with Roy Marquez. Developed a list of "attack" to tear apart and reassemble the following statements: Prove people gravitate towards maturity/expertise/learning/interactivity Why do novices (no expertise) work to be experts when experts do not work to be experts in two things Which is better for Prensky talk? Interactivity (exchange of information) or Desire for stimulus (senses) or effort? What is the truth in familiarity/expertise with regards to convenience or likelihood of use Immigrants purposed research is always related to their expertise(s)? Is expertise the reason Digital Immigrants don't become skilled with technology? Is "conditioning" relevant/correct? If dual encoding is a factor in the model (from 10/16), then TV should be more interactive than websites to utilize more senses? Keywords identified to discuss Prensky: active, broadcast, asynchronous, full sync, passive, engaged, interactive
10/18/2006	Pulled some ideas together regarding adult learning theory. Bruning, Schraw, Norby, & Ronning remarked that schemas are a source of predictions. I hypothesized that sub-schemas for "research" could be different for DIs and DNs based on their backgrounds of experience (or exposure). Metacognition was related to utilization of dual modalities. DNs know they can get dual encoding from electronics On-demand information means people don't have to remember as much anymore Noted that sensory registers are smaller in kids Good learners have good attention management as part of attention management theory Theory: DNs decode graphics easier, DIs decode text easier Theory: familiarity is directly related to expertise Noted to lookup Glaser and Chi (1988) "The Nature of Expertise" Added an element of power to the hierarchy of interactivity, though don't know if they're equivalent in terms of the ordering or presentation of the chart items Question: where do non-traditional learning environments fit in the hierarchy of interactivity (i.e. paintball, chess, golf)? Other things for interactivity hierarchy placement: TV, book , ITV, telephone, mobile phone, texting, IMs, chatroom, BB, Quake, lecture
10/20/2006	Found schema scripts in Bruning, Schraw, Norby, & Ronning. Received script book by Schank and Abelson. The script book is not what I expected and the diagrams inside are hard to read (as in understand). Only the first section of the book appears to be immediately helpful in what I'm working on.
10/22/2006	Added rewards and consequences to the hierarchy of intractivity. Found several resources that I added to my comprehensive exam regarding expertise.
10/24/2006	Presented the proposal presentation Powerpoint in class. Decided some aspects of my research may need to be reinvestigated regarding it possibly being a case study. Piloted my questionnaire. The participant showed the DI and DN names may not be completely appropriate for what Prensky discusses in his papers. Talked with Roy and Ben about doing peer review. Ben gave me a hard copy of his memo and a CD with the recording of his interview.
10/26/2006	Listened to Ben's interview and read his memo. He did a good job with the transcription, adding pauses in the text where appropriate. I noticed there were more pauses in his interview than in mine.
10/29/2006	Entered the transcript of my interview on my personal website. Before I published it, I realized IRB might disagree with me putting the contents on the Internet for the world to see, especially since my IRB document says all my research will be done anonymously. I didn't think there were any true identifying characteristics of my participant, especially if I replaced her name with Anonymous, but I opted to leave it unpublished just in case.
11/1/2006	Did all the final writeup of commentary about Ben's interview. Gave him some advice on how I might have handled parts of his interview differently. Would have been nice to read his reply to my reply instead of it just being submitted to the homework box.
11/4/2006	Submitted the response to Ben's comments about my memo to the WebCT homework box. It was interesting to note Ben thought some of my survey went wrong with in fact it went exactly the way I hoped in terms of information discovery. I'll have to note here to remind myself to tell him in class since I won't be sending him a copy of the Word document.
11/7/2006	Realized during class the title of my study says "students," but the listserv I used is not limited to students or even UCF. There is a good chance UCF IT professors answered the questionnaire as well as industry professionals who have never taken a formal academic instructional technology class also participated in the questionnaire. Dr. Hirumi has noted in the past at his monthly social meetings he encourages people all over the country to join the UCF IT listserv. As far as I know right now, it should only affect my project title.
11/10/2006	Started entering the Digital Propensity Index questionnaire on Zoomerang to prepare it as a possible route for validation. Called Zoomerang to ask how much it costs to purchase a survey audience if I didn't have one on my own. They have a 200 response guaranteed minimum and it costs $1350. Their 500 guaranteed responses from a random sample of general population is $2350. Not only was it too expensive, but they offer incentives for people to take surveys so I would have another thing to have to declare as possible reporting error if I published the results of any research using their population.
11/11/2006	Investigated Keller's ARCS model as a way of measuring Prensky's propositions. Discovered dissertations done at FSU under Dr. Keller that used surveys Keller developed and validated to measure instructional motivation with regards to ARCS. Keller has a manuals titled "Manual for Instructional Materials Motivation Survey (IMMS)" and "Manual for Course Interest Survey (CIS)". Was not able to locate them in the UCF library for interlibrary loan, on Google, or in EBSCO.
11/13/2006	Contacted Dr. Keller and Dr. Hirumi to get manuals on the Course Interest Survey and the Instructional Materials Motivational Survey for Keller's ARCS model. Keller didn't reply, but I met with Dr. Hirumi in person, who had a copy of the surveys, the reliability analysis results and some of the details on how to score the results of the survey. 2c thought they would be good surveys to use in my dissertation and that the documents he sent me would have enough details for what he expected to see in Chapter 3 of my proposal.
11/14/2006	Had class tonight. Dr. Blasi was serious about creating a physical poster for the presentation on the 28th. A Powerpoint uploaded to the homework box won't cut it.
11/16/2006	Bought a poster board at Walmart tonight. Got it home and it was too short. Won't be able to go find another one - going to shrink the text to get it to all fit on the board.
11/24/2006	Dr. Keller replied to my email. I just need to tell him where I'm going to school and who my advising professor is, then he'll send me the aforementioned ARCS documents.

Mini Study Analytic Memos [partial]

The Prensky stuff I have been doing has been planned to be researched on insurance industry professionals. When I piloted the survey instrument for the Digital Propensity Index to a sample of the professionals I had planned on surveying, I only had a 0.4% response rate compared to over 4% for undergrads, 10% for the graduate students, and over an estimated 30% for instructional technology cohort, using the same contact emails. It may be time to re-review the Dillman TDM for the 2007 update on doing surveys. I may also need to ask Howard at CHL Marketing in person if I could use his list of customers as contacts.

The idea behind piloting to the insurance professionals, even though I wanted to save them for my dissertation, was to make sure the DPI questionnaire was valid for them. Unfortunately, I probably will not make that discovery, nor will I get enough responses in my dissertation, using that audience and a quantitative method, to graduate. That may mean it is time to open my realm of considerations to a more mixed-method type of survey.

---

Dr. Hirumi forwarded an email to the Instructional Technology listserv about a researcher in the UK who is expanding on Prensky's research, and bringing it to the attention of government. Though I thought some of the comments in the article were made for flash and drama, it still had very little real evidence to explain the differences between Digital Immigrants and Digital Natives.

The article discussed having children get more exercise to release good chemicals in the brain. One of the points said children play games before going to sleep, which releases chemicals in the brain to keep them awake longer when they actually do try to fall asleep, then suffer in school from sleep deprivation. While that doesn't have anything to do per say with instructional technology or the design of instruction, it is a notable point of interest to not have your kids playing games right before they go to bed. In fact, I fail to see why any of the article is related to the design of instruction.

---

Preliminary findings appear to show comments on the following items related to technology, electronics, and digital toys:

Digital Immigrants	Mixed	Digital Natives
Telephones Use VCRs Simple computer functions Copy machines Library searches Typing Digital cameras Limited technological proficiency DVD players Basic features of mobile phones Limited knowledge of Microsoft Office Prefer traditional lecture Communicate face-to-face or by conference call Prefer instruction on paper	Internet navigation Word processing Forwarding of emails	Program VCRs LEET speak Instant messaging MySpace Shortcuts Use of icons and buttons Comfortable with electronics Change ringtones on mobile phones PDAs Blogging Play computerized games Use portable music players (iPods) Full functional knowledge of Microsoft Office Communicate by online chat Prefer web-based instruction Only meet face-to-face to formalize and finalize projects

A recurring theme is how Digital Immigrants use electronics for task completion, whereas Digital Natives additionally use electronics for entertainment.

One downside of interviewing people already potentially familiar with Prensky is how one person preferred to argue with how I asked questions and my use of "Digital Native" and "Digital Immigrant," rather than just answer as I had asked.

I noted one respondent age 60+ said "old school" college processors will deliver course materials as "force fed, instructor delivered, and stale." If older people are truly Immigrants, should they think instruction is stale, even if it is their preference?

Attachment	Size
Norman,David_Kent-Analytic_memos.pdf	557.43 KB

Mini Study Observation Interviews Documents

Instructions: Please title the pages with your name and project title and the IRB number. Please provide narration or written context so that an outsider from our class can look at it and understand what they are looking at (the purpose of the data collected) and what it means (why you collected it for your study, what you found from it).

Practice interview and memo

The real participant's name has been removed to protect anonymity.

The interview process has been related to something I have been working on researching for several semesters. Unfortunately, I have spent much of my discussion time during research talking with other instructional technology academics about the topic of Marc Prensky's Digital Natives and Digital Immigrants. The purpose of this interview was to talk with someone who I viewed as borderline between a Digital Immigrant and Digital Native so they could potentially see both sides of the characteristics between the two. Included is an analytic memo and transcript of the interview.

It also served as a pre-pilot to my mini-pilot study for EDF7475a Qualitative Research in Education. The questions were pre-written and approved by the University of Central Florida IRB committee. The informed consent document was read, for the most part, verbatim from the version the IRB approved, with exceptions for the obvious differences. The original informed consent document was designed for an anonymous, web-based questionnaire.

The overall goal of the questionnaire was get other thoughts about the accuracy of the adjectives Digital Native and Digital Immigrant, coined by Marc Prensky. Prensky's descriptions of each type of person are based in a large part on age, which is why Anonymous was a convenient participant, as closely randomly selected as possible. Generally, a Digital Native is named as such because they grew up with video games, mobile phones, the Internet, digital cameras, and so on, whereas Digital Immigrants did not have such technologies as children, so they have had to adapt to incorporate them in their lifestyle.

Before I started the interview, I expected Digital Natives to be more simply defined as a young person and a Digital Immigrant as and old geezer. I was actually concerned the answers to my questions would be so short and matter-of-fact, I wouldn't have much to follow up on. I even got a little adrenaline rush of fear when she asked what a Digital Native was. The interview instantly got a feeling of a difficult exam rather than a casual interview. I've taken plenty of tests where I just have no idea what the answer to a question is, so I was glad with Anonymous picked up and ran along with an answer. It was almost like she was thinking aloud, which was great for extracting information from the interview process. She broke the parts of the terminology down to things she was familiar with.

At times, in the interview, she seemed to stray from the topic to tell some sort of personal story I didn't think at the time had much to do with the interview questions. In reflection, I could have probably asked follow-up questions to tie her personal story back to the question and make her talk even more and faster. For example, Prensky suggests Digital Immigrants won't be as successful as Digital Natives unless they make an effort to become Digital Immigrants, as opposed to being digitally ignorant. When Anonymous started talking about her Mexican neighbor, I might have been able to tie that in to success; for example, something along the lines of, "How do you think not learning English affected your neighbors' success in life?"

I feel like there should be a set of follow-up questions related to Anonymous's line of thinking about resistance to new technology, then saying Digital Immigrants should make multi-dollar business out of it. That might have better helped define the business relationship she was putting on the differences between Digital Natives and Digital Immigrants.
As the interview went on, the conversation turned from an awkward series of pauses, to something more like a free-flowing brainstorm. Getting to the brainstorm stage of the interview, and so quickly, made it kind of disappointing for the interview to end, especially since I have so much of this topic riding on my dissertation.

In terms of inferences, she thought Digital Immigrants were hard-working, self made business owners. Digital Natives were more likely to follow a structured, incremental series of promotions through a corporation. She saw how not every culture or country is digitally fluent. She agreed, in part, with Prensky to say older people would prefer to do things in a more face-to-face manner, whereas a younger person would prefer Internet-enabled methods of communication.

Trying to interview in a noisy restaurant, while your participant is trying to eat, with a limited lunch hour may not have been the best environment for me to have done this interview. The time constraint for their lunch period may have kept her answers shorter than they could have been. That was especially evident at the end of the interview when she politely asked if we could wrap up the interview. Also, asking someone I didn't know makes it hard for me to ask follow-up questions later on. However, on the other hand, the questionnaire and IRB approval are all based on a matter of anonymity in the first place, so I have a better understanding for why a researcher might go through all the extra effort to make a study confidential as opposed to anonymous in nature.

I did not take any notes during the interview. I was trying so hard to keep her attention with eye contact; I didn't feel comfortable with recording and writing things. In retrospect, I probably should have just asked her if doing both was alright with her.

 

1	DKN: Thank you for agreeing to be interviewed today, so I can practice my interviewing for class. The participant asked to be identified as simply Anonymous. (.) This interview should take 15 or 20 minutes. Our location is the Panera Bread (.) on uh, (.) in Waterford Lakes. (.) [shuffling paper] I'm going to read a document to you that is important for your own protection.
2	Ok. (.) What is it?
3	DKN: The school has a procedure (.) er (.) office or whatever called IRB that's supposed to monitor the studies done at the university to make sure nobody is hurt in any way during research projects. (.) I'm going to read the (.) um (.) disclaimer to you that they had me submit for approval. Ready?
4	Yes.
5	DKN: Ok. Just stop me if at any point you don't understand or agree with something (.) I am a graduate student in Instructional Technology at the University of Central Florida. I am conducting a pilot study this fall, the purpose of which is to determine perceptions of how Instructional Technology students think age affects proficiency with using electronics. (.) This research study has been approved by the Institutional Review Board (IRB) at the University of Central Florida (UCF). (.) Your interview will be conducted in via an online survey service. The link is below. The questions will be predetermined. There will be a free response area at the end of the survey where you may suggest topics and opinions in any area you feel is relevant to the topics presented. The survey may be completed at your convenience. (.) In your case, your responses will not be anonymous or confidential. They will she shared with the rest of the students in my class and my professor, and might be used in any papers I publish in the future, even though I'm not quite sure if I would actually include this interview. (.) There are no (.) anticipated risks, compensation or other direct benefits to you as a participant in this interview. The survey should take approximately 15 minutes to complete. You are free to withdraw your consent to participate and may discontinue your participation in the interview at any time without consequence. If you have any questions about this research project, please contact my faculty supervisor, Dr. Laura Blasi at: 407/823-1761. Information regarding your rights as a research volunteer may be obtained from: (.) Barbara Ward, Institutional Review Board (IRB) University of Central Florida (UCF) (.) 12202 Research Parkway, Suite 501; Orlando, Florida 32826-3252 Telephone: (407) 823-2901 (.) If you decide to participate in this research study, you must be at least 18 years old and give consent that I can get for archival on this recording. If you agree, go ahead, say you're over 18, say your name, and say you agree.
6	My name is Anonymous, I'm well over 18, and I'm more than happy to take yer survey.
7	DKN: Ok, then. (.) Question 1. (.) Based just on the label "Digital Immigrant", what skills would you imagine such a person having with electronics?
8	Uh (.) A what?
9	DKN: A Digital Immigrant. If I pointed at someone and called them a Digital Immigrant, what skills would you think they had with electronics?
10	I guess (.) since it has the word immigrant in it (.) they wouldn't be very experienced with them (.) but then even immigrants have some sort of exposure to a new land. I guess by that you mean someone who is learning electronics so they can adapt to a new environment. I always think of New York when I hear the word immigrant and pictures I've seen of immigrants getting off boats to see the US for the first time. With that mental picture I see someone who wants to learn how to be a US citizen, or in this case, someone who wants to learn how to use electronics, but has kind of an old accent of their old ways. You know (.) as I think about it more (.) there are also Mexican immigrants who I also have noticed don't necessarily even try to learn English. I had a neighbor once who's wife didn't even know English. We just waved at each other in the driveways and I talked to the husband when he was home since he worked in retail. So with that picture, maybe this Digital Immigrant person is resisting new technologies? Could you clarify it for me?
11	DKN: Well, (.) I already have an idea of my own. (.) I'm more interested in what you think about it.
12	Alright fine. Uhm (.) In that case, (.) I'm going to hope you have a positive outlook on life (.) and I'll go with the New York images. Then a Digital Immigrant is someone who sees all the new electronics out in the world and wants a piece of the action. I bet it doesn't take them long to be that guy who owns Blockbuster and the football team, who started from nothing, worked hard, and made a multi gazillion dollar business out of it.
13	DKN: Alright (.) I think that covers that question. I'm very curious to hear about the next topic. So question 2. Based solely on the label "Digital Native", what skills would you imagine such a person having with electronics?
14	I had a feeling something like that was coming. [chuckle] I don't think a Digital Native would have an appreciation for the things available to them. Cell phones are just a part of their normal, daily life. They just expect email to be there and work. I think of entitlements when I think of natives, but I'm not sure what exactly is an electronic entitlement. I spose the Internet could be an entitlement. It's just always there, it's the basis for a lot of communication and business. They probably get their news off the internet instead of from the TV. Since it's probably just something normal to them, they wouldn't even realize how cool it is to be able to get an old book off the Internet from a library in another continent. They wouldn't have the same attitude to do cool stuff with it like an immigrant would.
15	DKN: Alright (.) now describe how the formal education of a "Digital Immigrant" might differ from a "Digital Native."
16	Why does it have to differ? Can't they have the same background?
17	DKN: Well, like I said before, I already have thoughts on the matter. I'll back up then. Do you think a Digital Immigrant would have different formal education than a Digital Native?
18	I don't know why they would. Someone in their 20's could get a Ph.D. just the same as someone who's had their Ph.D. since 1960 and they could be in different countries from different colleges of learning, like business and biology. (.) I spose they could have different ways of having done their homework. The old guy probably had to use a card catalog on the library whereas the young one searched Google all the time and complained about their professors in a blog somewhere. (.) You know, I complained about a professor online once and they actually found it and confronted me about it. I don't recommend it. At least wait until after you graduate. By then, you probably won't care about complaining anymore.
19	DKN: I guess I may need to revise that question in the future.
20	Yeah, I think so.
21	DKN: Next, describe how you think a 19-year-old college Freshman would prefer to receive their course materials.
22	I bet they want everything by email. They probably just want to talk to their professor over Yahoo Messenger or something. You know, I bet all the new students in college that bring laptops to class don't even listen to the lectures. I can see them surfing the internet, checking their email, blogging if that's what it's still called, downloading the newest and coolest new games. Kids are always talking about how bored they are or how tired they are. They probably want to just sleep in till noon every day, goof off in front of the TV, and get to the school junk whenever it's more convenient. Maybe they open an email in the morning and decide not to work on it till after dinner.
23	DKN: Alright, now describe how you think a 19-year-old college Freshman would prefer to collaborate on a group project versus a 60-year-old retiree.
24	First, I bet the retiree is the one that takes over the group leadership. They probably try to setup a face-to-face meeting somewhere, exchange phone numbers, addresses, email information. Meanwhile, the 19-year-old is probably thinking to themselves that they just want to meet online. You know, I've heard that before, but I've never really quite understood what it means to meet online. Do people really set up cameras and try to have a screen full of other people's cameras? My thinking outloud is probably driving you nuts, huh.
25	DKN: Absolutely not. You're actually being quite helpful.
26	I don't know how, but anyway. The freshman probably wants to write some Office documents and fire them back and forth via email. The old guy probably wants to meet at the library in a study room and draw on the caulk board.
27	DKN: I think that will do for that question. Next, describe how you think a college professor would prefer to generate and present lessons to 19-year-old college freshmen.
28	Powerpoint seems like it would be a popular choice. I bet you can even get slides from the book publishers now so you don't even have to create the content yourself. Even if they can't get them from the publishers, they probably copy a bunch of text from the textbooks that the students don't read anyway then read the Powerpoints to the students during class. I guess if they were a really old professor, they could have some overheads or something.
29	DKN: Can you give an example of when a technology boundary negatively affected your success on a project for work or school?
30	I procrastinated on an assignment once. It was for a web class where we had to use webct and upload the final work to the homework dropbox thingie. I was ready to turn it in at the last minute, but my internet connection went down. By the time I called Bellsouth, got everything working again, the dropbox thing was closed. I ended up having to email the assignment through webct email, which the professor never checked, and I had a bunch of headaches explaining why the assignment was late, the professor complaining about it not being turned in, in a way that made it organized for them to grade papers, and. It was just a bad experience. I ended up getting 20 points knocked off the assignment, which still makes me mad because I had the work done, it was just my connection and the professor didn't believe me. I did learn not to procrastinate quite as much, though.
31	DKN: Can you give an example of when you perceived age was a boundary in the success of a project at work or school?
32	I always like to think I'm smarter than my boss. My boss told me once, though, that I wasn't old enough to have the experience to handle being promoted. I sure didn't like that answer since I thought I was smarter than him in the first place. Who knows, maybe I said something to piss him off. Needless to say, I don't work there anymore or talk to that jerk. I need to go. Can we wrap this up?
33	DKN: Sure, I just have some short questions left. I'm going to read some age categories. Tell me which one yours falls in. Under 25, 25 to 34?
34	That one.
35	DKN: 25 to 34?
36	Yes.
37	DKN: What is your highest educational degree?
38	I have a BA.
39	DKN: Please, could you suggest topics and/or opinions in any area you feel is relevant to the topics presented in this questionnaire.
40	Not really. I need to go.
41	DKN: Alright then. (.) Uh. (.) I appreciate the time you gave me.
42	No problem. (.) Good luck with the rest of your assignment.