ASSESSMENT AND TESTING – MASTER ASSIGNMENT

Introduction:

This is an actual end of unit progress test from a project I am working on. The project is an on-line General and Technical English course. The course consists of fifty units which takes learners from elementary to advanced level. I have chosen progress tests from Unit 37 which is intermediate level (Common European Framework level B2). I have included two tests because the learners I work with do the general and technical English courses in parallel.

Each unit of the course has seven sections:

1. Introduction

2. Vocabulary

3. Reading

4. Structures

5. Functions

6. Listening

7. Communication Activities

8. Self Evaluation

1. The INTRODUCTION section serves two purposes in that it reviews the previous unit and introduces the new one.

2. The VOCABULARY section introduces the lexical element of the unit.

3. READING skills are practised in this section

4. Grammatical STRUCTURES are covered in this section.

5. Language FUNCTIONS are practised.

6. LISTENING skills are covered in this section.

7. Learners take part in communicative activities in the COMMUNICATION ACTIVITIES section.

8. In the SELF EVALUATION section learners decide for themselves if they are able to move on to the next unit or if they need to go over the unit again or require help from their tutor.

The learners do the course at home or at work on-line. They have access to a virtual classroom and are in touch with fellow learners and a tutor by e-mail and chat.

Common European Framework of Reference for Languages

The European Language Portfolio.

The institution where I teach has given a commitment to work to implement the ELP. Therefore the test I am presenting for this assignment has to be seen in this context. It is an attempt to introduce self-evaluation and learner responsibility and autonomy which are at the heart of the ELP approach. However we have to work in the language teaching world as it is at the moment and so my test contains aspects of older forms of testing as well as the new approach of the ELP.

I feel it would be useful to summarize the main objectives of the ELP.

  • Promotion of multilingualism and cross-cultural dialogue.
  • Facilitation of mobility in Europe.
  • Strengthening and preservation of cultural diversity.
  • Promotion of autonomy in learning.
  • Encouragement of life-long language learning.

The ELP also has three main aspects:

  • The Language Passport gives an overview of the current level of language proficiency and summarizes the learning and intercultural experiences of its holder.
  • The Language Biography documents the personal history of language learning and intercultural experiences. It also contains instruments for self-assessment of language proficiency and helps reflection and planning of learning.
  • The Dossier is a collection of work that illustrates what the learner has done and is able to do in different languages. On the one hand the dossier can be used as a “working dossier” which accompanies daily language learning and documents the learning process, on the other hand as a “showcase dossier” which illustrates the present level of language proficiency.

THE TYPE OF TEST ACCORDING TO PURPOSE?

The first point to be made about the type of test I have designed is that it needs to be seen in the overall context of the course. The on-line course aims to be communicative and interactive. However it must be borne in mind that the learner is physically isolated from and never meets their fellow learners or tutor. Therefore the end of unit test or ‘Self Evaluation Test’ has to serve multiple purposes.

The three main purposes are:

1. As a Progress Test: Most normal classroom tests take this form. They assess progress students make in mastering material taught in the classroom. They are often given to motivate learners. They also enable learners to assess the degree of success of teaching and learning and to identify areas of weakness and difficulty. Progress tests can also be diagnostic to some degree.

However, as this is an on-line course it takes a slightly different form. The tutor receives the data from the learner’s Self Evaluation test but there is nothing to stop the learner ploughing on to the next unit / level even if they have scored quite low in the test. If the tutor sees that the learner is consistently scoring low in the tests they should point it out to the learner and offer counselling but the idea is that the learner themselves should take responsibility for monitoring their own progress and asking for advice if they feel it is needed.

2. I would also say that the Self Evaluation test should be considered as a Diagnostic Test as well. Diagnostic tests enable teachers to identify specific weaknesses and difficulties so that an appropriate remedial programme can be planned. With my Self Evaluation Test the emphasis is on the learner to discover their own weaknesses and seek assistance and solutions. They are also encouraged to recognise their strengths and build upon them.

3. The final purpose of the Self Evaluation Test is to raise awareness of learning strategies and promote learner autonomy. The advice and tips given at the end of the test are devices to encourage learners to take an interest in the processes of learning a second language, in effect, to learn how to learn.

TEST GENERATION?

I would not consider any aspect of my test to be first generation. Questions 17 and 18 in the General English section could be interpreted as first generation if looked at superficially but in reality they are real-world tasks and involve a degree of peer correction. The questions are testing the communicative ability of the learner and as such are not open to the subjective judgement of an examiner which is the hallmark of first generation testing.

I would consider questions 6 – 16 in the General English section and all the questions (apart from Q9) in the Technical English section to be Second Generation type questions. They test discrete items including pronunciation, vocabulary and grammatical structures. The questions relate to specific items which have been taught in the module.

Overall I would consider that my test is an example of a Third Generation Test. I have tried to contextualize the questions and make them as real-world as possible. There are examples of integrative questions and questions which aim to test global comprehension. (Q17 & Q18 General English and Q9 Technical English) These types of questions also test the productive skills in that the learner has to write and speak and exchange information with the tutor and other learners. I have tried to create a test which both tests the learner’s language knowledge and the capacity to use that knowledge and language in communicative situations.

I would also add that my test is an attempt to integrate the self evaluation and learner autonomy / responsibility components of the European Language Portfolio. There is a clear attempt in the test to edge the learner towards self assessment and to think about and understand the language learning process. One of the aims of my test is for the learner to realize that the teacher is merely a guide and for a successful outcome of the language learning process the learners themselves have to assume a greater responsibility. They are encouraged to diagnose problem areas, set objectives and assess their learning styles and progress.

PRINCIPLES OF LANGUAGE TESTING?

As I have explained above my test is a combination of Second and Third Generation Tests and Self and Peer Evaluation. I will comment on each of the pairs of principles.

COMPETENCE VS. PERFORMANCE

I feel there is a balance in the test between finding out what the learner knows about the language and how they use it in concrete communicative situations.

USAGE VS. USE

Again there are examples of both in my test. The learners are asked to give examples of vocabulary and structures to indicate their knowledge of the rules (usage) but they are also asked to produce language in performance (use).

DIRECT/INDIRECT ASSESSMENT

Although there are examples of indirect testing, I have endeavoured to make my test a direct test overall.

DISCRETE POINT VS. INTEGRATIVE ASSESSMENT

As I have explained above my test includes examples of both discrete item and integrative assessment.

OBJECTIVE VS. SUBJECTIVE ASSESSMENT

Again the test contains examples of both objective and subjective assessment. The subjective assessment centres on the productive skills. In an attempt to overcome the usual problems with subjective assessment, peer assessment and self-evaluation have also been introduced.

RECEPTIVE VS. PRODUCTIVE SKILLS

The test attempts to assess receptive and productive skills.

BACKWARD AND FORWARD LOOKING ASSESSMENT

In general I would say this test is more forward looking than backward. Although it tests what was taught in the module it also attempts to assess the use of real-world language and so in that sense it is definitely forward looking.

CONTEXTUALIZED VS. DISEMBODIED LANGUAGE

The whole course I have written attempts to use contextualized language. Great care has been taken to give a context to all activities and tasks. (See above – ‘Communication Activity’) Therefore the Self-Evaluation Test aims to reflect this.

CRITERION REFERENCED AND NORM-REFERENCED ASSESSMENT

The main idea behind the test is to introduce the learners to Self-Evaluation as defined by the ELP. The test is therefore certainly not Norm-Referenced. It leans more towards Criterion Referenced assessment but with the emphasis being on the learner themselves to make the judgement whether or not they have successfully completed the various tasks.

RELIABILTY VS. VALIDITY

Overall the test is subjective in that the majority of the questions or reflections are not objective. This should in theory reduce reliability but I strongly feel that the self-evaluation and peer evaluation element actually increases the validity. Initial studies on the implementation the ELP show that self-evaluation is extremely accurate.

Content Reliability: In my opinion the test reflects perfectly the language of the syllabus.

Construct validity: My test not only is mainly a third generation test but is an attempt to promote autonomy in learning and self-evaluation, which are both current theories in language learning and testing.

Predictive validity: I think test does assess future language performance quite accurately, especially the LEARNING DIARY reflection at the end of the test.

Concurrent validity: I am confident that my test would produce similar results to established tests.

The underlying principle in my test is self-evaluation. I believe self-evaluation enables learners to develop their critical faculties. Self-assessment also enables learners to look at language in more concrete terms. Self-assessment hopefully will motivate learners to look at their strengths and weaknesses and become more autonomous learners which is a fundamental part of the learning process.

According to Oscarsson (1989), there are six different reasons why self-assessment can be beneficial to language learning.

1. Self-assessment promotes learning, plain and simple. It gives learners training in evaluation which results in benefits to the learning process.

2. It gives both learners and teachers a raised level of awareness of perceived levels of abilities. Training in self-assessment, even in its simplest form, like asking “What have I been learning?” encourages learners to look at course content in a more discerning way.

3. It is highly motivating in terms of goal-orientation.

4. Through the use of self-assessment methodologies, the range of assessment techniques is expanded in the classroom. As a result of using self-assessment, the learner broadens’ his/her range of experience within the realm of assessment.

5. By practicing self-assessment, the learners participate in their own evaluation.

They, in effect, share the assessment burden with the teacher.

6. By successfully involving learners in their own assessment, beneficial post-course effects will ensue.

RELEVANT SCALES AND INSTRUCTIONS FOR MARKING TEST?

Overall the test is based on the ELP self-assessment common reference levels. The global scales are as follows:

Basic User:

A1

A2

Independent User:

B1

B2

Proficient User:

C1

C2

There are also specific questions which relate to the material covered in the module which are backward looking but overall the test is an attempt to introduce self-evaluation in the context of an on-line course.

The self-evaluation is achieved by means of three devices:

Rating Scales

Check lists

Questionnaires

These three techniques have been used in self-evaluation. They have been used as a means where learners can rate their perceived general language proficiency or ability level. A lot of developmental work has been done in this area through the use of “ability statements” such as “I can read and understand newspaper articles intended for native speakers of the language.” I have adapted the statements in my test from ELP materials.

The learners are instructed to indicate their estimated ability to cope with situations by reflecting on and ticking the described level of performance. (In the web version of my test the learner can tick a check box to indicate if they agree with the self-evaluation statements. This is not possible to show in this printed version.)

TEST FORMATS?

The test uses three main formats.

1. The questions relating to the material taught in the module (mainly grammar and vocabulary) are in a multiple choice format.

2. The self evaluation is achieved by means of statement ‘Check lists.’

3. Communicative competence is tested authentically where possible and is interaction based, unpredictable, and varied in context and has a communicative purpose.

These formats have been used because I feel they are most appropriate to the type of test I wanted to create.

UNIT 37 GENERAL ENGLISH JAM SESSION

After finishing the unit can you…

  1. express yourself using similes and comparisons?
    1. Yes
    2. No
  1. distinguish if a word has the sound /s/ or /z/?
    1. Yes
    2. No
  1. name at least four similes using ‘like’ or ‘as’ and name several words related to music?
    1. Yes
    2. No
  1. use different words and expressions that have either positive or negative meaning?
    1. Yes
    2. No
  2. review a film, a song or a book?
    1. Yes
    2. No

Listen to the following words and click on the sound they have.

  1. a. /z/ sound

b. /s/ sound

  1. a. /z/ sound

b. /s/ sound

  1. a. /z/ sound

b. /s/ sound

  1. When my sister wants to go out she gets dressed as ________ as a flash.

a. cold

b. quick

c. stubborn

  1. I’ve been eating like a _______ for the last week because I’ve been really nervous.

a. bear

b. fish

c. bird

  1. Yesterday a man came to the surgery and he was as deaf as a ________.

a. post

b. clock

c. log

12. Which is the odd one out?

    1. Tacky
    2. Cool
    3. Rubbish
  1. What does ‘haven’t got a clue’ mean?
    1. that you are an expert at something
    2. that you have no idea about something
    3. that you haven’t got an opinion about something
  1. Click on the right option.
    1. a leather pair of comfortable shoes
    2. a pair of leather shoes comfortable
    3. a comfortable leather pair of shoes
  1. I think James Joyce books are extremely boring and difficult to read.
    1. gradable adjective
    2. un-gradable adjective
  1. Yesterday I met an English man and he spoke to me…
    1. as slowly like he could.
    2. like slowly like he could
    3. as slowly as he could

Writing an E-mail

17. Try to remember the best concert you have ever been to. Write an e-mail to your study-mate telling them about the event. Try to use some of the words you have learned in this unit. Use comparisons and adjectives. Send me a copy of the message as well.

Reply to any messages you receive from study mates.

Leaving a voice mail message.

18. Your favourite group are playing in your town. Send a message to your study-mate telling when and where the concert will be and how much the tickets will cost. Try to persuade your study-mate to go with you. Send me a copy of the message as well.

Reply to any messages you receive from study mates.

Pronunciation & Vocabulary

19. You should record all the new vocabulary in your Audio Diary and check your pronunciation with the model pronunciation in the Audio Dictionary.

GENERAL ENGLISH FEEDBACK

POINTS TO THINK ABOUT

0-6: Perhaps you need to work more on some points covered in this unit. This could be because you’ve skimmed over some sections, or because you’ve had trouble concentrating. Have a break and go over the things you aren’t sure about again. Remember – this part of the unit is to help you consolidate what you’ve learned.

7-11: Well done – you’ve passed the test. However, you might need to look through some sections again. You’ve obviously understood a large part of this unit, but you should make sure you are clear about all aspects of this part of the course. Don’t go on to the next unit until you are clear about this one.

12-16: Congratulations! You’re doing very well so far. If you want more things to study, have a look at the other sections in this course, by clicking on comunidad. If you are interested in studying abroad, click on servicio lingüístico. Keep on working hard and good luck!

17 & 18: Make sure you send the e-mail and voice message. I’ll get back to you about them as soon as I receive them. Don’t forget to reply to any messages you receive.

19. Did you check your pronunciation? How does it compare to the model pronunciation in the Audio Diary. Get back to me if you have any doubts. You can record any words you are not sure about and send them to me in a sound file and I’ll let you know my opinion. You can also send the same files to your study-mates to see what they think as well.

UNIT 37 TECHNICAL ENGLISH – SOUND TECHNOLOGY

It’s that time again – The Self Evaluation Test. You can only do your best!

1. Which word is the odd one out?

    1. amplified
    2. amplifier
    3. noisy

2. What does ‘faint’ mean in relation to sound?

    1. That a sound is very loud
    2. That a sound is of low intensity
    3. that a sound is silent

Listen carefully:

3. What is the sound? Click on the right option.

a. Loud

b. Low

c. Fuzzy

Listen carefully:

4. What’s wrong with this cassette?

a. It’s fuzzy

b. It’s faint

c. There’s no bass

What’s the word?

5. A device that uses lasers to produce sound

a. stand

b. CD player

c. speaker

6. A device which converts electrical energy into sound waves

a. stand

b. a cassette deck

c. speaker

7. Surround sound is…..

a. spatial

b. spacey

c. special

8. Two optical tracks on the film to create four distinct channels of sound

a. Dolby

b. Matrixing

c. THX

Writing an E-mail

9. Electrical equipment that didn’t break would be like a dog that didn’t bark. Stereo systems, of course, are no different and every major retail store has a customer service department where broken or faulty equipment can be returned for repair.

ACTIVITY ONE

You’ve just accepted a job in a shop which sells audio equipment. It’s your first day at work and you’ve been left in charge of customer services. There are five pieces of faulty equipment. In order to make sure everything is repaired correctly you have to package the broken items and enclose a note explaining what you think may be wrong with each system. But all you’ve got to guide you are the sounds of the broken equipment, and some comments from their owners. Click on each picture to hear what’s wrong – and DO read the comment accompanying it. Then select the best word to describe the problem.

(Learners click on buttons to hear 5 different sounds indicating what is wrong with machine.)

Instructions for the tape The recordings have the following specifications of sound.

Recording 1- Is fuzzy and not clear

Recording 2- Lacks some bass

Recording 3- Has a lot of background noise

Recording 4- Has almost no sound at all and is very faint

Recording -5 Very loud

1. Model Number: Sanyo 635

Customer complaint: When I put on a CD it sounds a bit funny.

2. Model Number: Marantz pm-57

Customer complaint: There don’t seem to be any low frequency sounds.

3. Model Number: Harman HD 710

Customer complaint: When I use the head phones there is a lot of hissing in the background.

4. Model Number: Sony SY-767

Customer complaint: The speakers seem to be of very low intensity. In fact I am not sure they work at all. Do you think the sound has been switched off ?

5. Model Number: Panasonic PN323

Customer complaint: When I listen to anything on the tape deck, CD player or radio, it hurts my ears and I think it may cause damage to my ear drum.

Activity Two

Now that you’ve worked out what the customer is trying to say, you have to package and write a short e-mail to the repair man, Jim*. Quote the model number and summarise what the problem is with each system.

* Jim can be one of your study-mates.

  • You will probably receive an email commenting on a similar problem from another study-mate. If there is anything you don’t understand in the e-mail you receive write a reply and send a copy to me.

Pronunciation & Vocabulary

10. You should record all the new vocabulary in your Audio Diary and check your pronunciation with the model pronunciation in the Audio Dictionary.

TECHNICAL ENGLISH FEEDBACK

POINTS TO THINK ABOUT

1. b Amplifier is a machine which makes sounds louder. The other two are adjectives which describe sound. If you didn’t know that revise the Vocabulary and Language Functions of this unit.

2. b If you didn’t know the answer was “of low intensity” have a look at Language Functions again.

  1. a If you didn’t know the answer and can still hear go to the Listening Section again and the Language Functions.
  1. c If you didn’t know there was no bass on that tape go back to Communication Activities and listen to the faulty cassettes.
  1. b Go and look at the Vocabulary section again if you weren’t sure about that.
  1. c As above!
  1. a You’ll have to look at the Reading section again if you didn’t know that surround sound comes from spatially distinguishable sources.
  1. Dolby did indeed invent this system but it’s called “Matrixing.” If you guessed the right answer okay…. But why not go back and look at the Reading section again?
  1. Activity 1

a) There isn’t any bass

b) The sound is very faint.

c) There is a lot of background noise.

d) The sound is very fuzzy.

e) The volume control button is broken and everything is so loud it’s deafening.

Activity 2

Reflect on any replies you receive from study-mates. If you have any questions for me do not hesitate to get in touch.

10. Did you check your pronunciation? How does it compare to the model pronunciation in the Audio Diary. Get back to me if you have any doubts. You can record any words you are not sure about and send them to me in a sound file and I’ll let you know my opinion. You can also send the same files to your study-mates to see what they think as well.

FINAL REFLECTION & FEEDBACK

Learning Diary:

We are now coming to the end of this level (B2) It is time to reflect on your overall progress. Please read these summaries of what a typical learner should be able to do at your level. Think about both General English and Technical English.

Record your responses in your learning diary. If you think you need to talk to me about anything we can arrange a chat. If you feel you are ready to move onto the next level after doing this reflection please let me know.

UNDERSTANDING

Listening: I can understand extended speech and lectures connected with my profession and follow even complex lines of argument provided the topic is reasonably familiar. I can understand most TV news and current affairs programmes. I can understand the majority of films in standard dialect and technical videos about my work.

Reading: I can read articles and reports concerned with contemporary problems in which the writers adopt particular attitudes or viewpoints. I can also read most technical reports and articles pertaining to my profession. I can understand contemporary literary prose.

SPEAKING:

Spoken Interaction: I can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible. I can take an active part in discussion in familiar contexts, accounting for and sustaining my views. I can take part in business and technical meetings and am able to negotiate.

Spoken Production: I can present clear, detailed descriptions on a wide range of subjects related to my field of interest and profession. I can explain a viewpoint on a topical issue giving the advantages and disadvantages of various options.

WRITING:

I write clear, detailed text on a wide range of subjects related to my interests. I can write an essay or report, passing on information or giving reasons in support of or against a particular point of view. I can write e-mails and faxes. I can write letters highlighting the personal significance of events and experiences.

C1 LEVEL – YOUR NEXT LEVEL

This is the profile of a proficient user of the language at the next level. Read the profile and think about whether you are ready to attempt to move to this level. Think about what you need to do to achieve a similar profile.

<C1 – Can understand a wide range of demanding, longer texts, and recognise implicit meaning. Can express him/herself fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organizational patterns, connectors and cohesive devices.>

Further Self-Evaluation

If you want to self-analyse your language learning abilities in more depth go to this page:

DIALANG

http://www.dialang.org/english/

You will need to download the software but it is well worth it. Let me and your classmates know how it goes.

BIBLIOGRAPHY

Self-assessment checklists from the Swiss

version of the European Language Portfolio (2004)

http://culture2.coe.int/portfolio/documents/appendix2.pdf

The EAQUALS-ALTE European Language Portfolio (2004)

http://www.eaquals.org/about/portfolio.asp

DIALANG

http://www.dialang.org/english/

Cohen, Andrew D (1994) Assessing Language Ability in the Classroom. Boston: Heinle & Heinle

http://www-writing.berkeley.edu/TESL-EJ/ej03/r12.html

Oscarsson, M. (1984). Self-Assessment of Foreign Language Skills: A Survey of Research and Development Work. Strasbourg: Council of Europe.
Oscarsson, M. (1997) “Self-Assessment of Foreign and Second Language Proficiency”. In The Encyclopedia of Language and Education, Vol. 7. Kluwer Academic Publishers, pp 175-187.
Oscarsson, M. (1998). “Learner Self-Assessment of Language Skills”. IATEFL TEA SIG Newsletter, Nov. 1998

About these ads

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: