Developing Web-Based Language Tests

Wojciech Malec
ISBN: 978-83-8061-641-7
Stron: 440
Format: B5
Rok wydania: 2018


TABLE OF CONTENTS

Acknowledgements

List of abbreviations

Introduction
Language testing and technology
Basic terms and concepts
Organisation of the book

PART I Principles of Assessment

Chapter 1
Test development
1. Introduction
2. Components of test development
2.1. Design
2.1.1. Test context specifications
2.1.2. Test structure specifications
2.1.3. Task specifications
2.2. Production
2.2.1. Task development
2.2.2. Test assembly
2.2.3. Pre-operational testing
2.3. Operational use
2.3.1. Administration
2.3.2. Scoring
2.3.3. Decisions and reporting
2.4. Evaluation
2.4.1. Collecting evidence
2.4.2. Analysis
2.4.3. Item banking
3. Summary

Chapter 2
Test evaluation
1. Introduction
2. Practicality
3. Authenticity
4. Reliability
4.1. Classical test theory
4.1.1. Standard error of measurement
4.2. Generalizability theory
4.2.1. One-facet crossed designs
4.2.1.1. Relative and absolute error
4.2.1.2. Coefficients
4.2.1.3. Coefficients and test length
4.2.1.4. Phi lambda
4.2.1.5. Standard error of measurement
4.2.1.6. GT-1 calculator
4.3. Decision consistency
4.3.1. Threshold loss agreement
4.3.2. Squared-error loss agreement
4.4. Interpreting reliability estimates
5. Validity
6. Impact
7. Validation
7.1. Validating classroom-based language tests
7.2. Conclusion
8. Summary

Chapter 3
Test items
1. Introduction
2. Items and tasks
3. Test formats
3.1. Selected-response items
3.1.1. Multiple choice
3.1.2. Binary choice
3.1.3. Multiple response
3.1.4. Multiple-choice cloze
3.1.5. Matching
3.1.6. Other types
3.2. Limited-production items
3.2.1. Gap-filling
3.2.2. Cloze and C-test
3.2.3. Gapped sentences
3.2.4. Transformation
3.2.5. Sentence writing
3.2.6. Error correction
3.2.7. Other types
3.3. Extended-production tasks
4. Choosing the item format
5. Summary

Chapter 4
Item analysis
1. Introduction
2. Item facility
3. Item discrimination
3.1. Norm-referenced testing
3.2. Criterion-referenced testing
3.3. Item discrimination and test reliability
4. Distractor evaluation
4.1. Criterion-referenced testing
4.2. Distractor evaluation criteria
5. Summary

PART II Web-Based Testing

Chapter 5
Technology in language testing
1. Introduction
2. Computers and the internet in language testing
2.1. Models of administration
2.2. Web-based testing
2.3. Strengths and limitations
3. WebClass
3.1. Administration
3.2. Communication
3.3. Materials
3.4. Assessment
3.5. Teaching and testing with WebClass
4. Summary

Chapter 6
Test design and production on WebClass
1. Introduction
2. Design
3. Production
3.1. Writing items and tasks one by one
3.1.1. Test items
3.1.1.1. MC variations
3.1.1.2. Other selected-response items
3.1.1.3. Limited-production items
3.1.2. Extended-production tasks
3.2. Converting text into test items
3.2.1. Selected-response items
3.2.2. Limited-production items
3.3. Importing previous items
3.4. Automated test assembly
4. Summary

Chapter 7
Test use and evaluation on WebClass
1. Introduction
2. Test use
2.1. Administration
2.2. Scoring and verification
2.2.1. Automated scoring of test items
2.2.2. Score verification
2.2.3. Scoring extended responses
2.3. Reports and feedback
3. Evaluation
3.1. Collecting evidence
3.2. Quantitative analysis
3.3. Item banking
4. Summary

Chapter 8
Administration mode effects
1. Introduction
2. Score comparability
3. The study
3.1. Method
3.1.1. Participants
3.1.2. Materials
3.1.3. Procedures
3.2. Results and discussion
3.2.1. Measurement characteristics
3.2.2. Comparability of score-based decisions
3.2.3. Individual differences
3.2.4. Follow-up study
3.2.5. Qualitative analysis of attitudes
3.3. Limitations
4. Summary

Conclusions

References

Appendices
Appx 1. Scoring a test item (questionnaire results)
Appx 2. Big Test 1 (PBT)&nbsp
Appx 3. Big Test 1 (WBT)

Index