- How did the data collection process go?
- What are some of the challenges you’re facing in data collection and analysis? What solutions have you devised?•
- Do you have any additional questions on coding, analysis or the project itself? (These will be shared with Jonas.)
The collection of my data went well so far, I have one more to do next week. No technical problems so far. I haven't started coding yet so I don't have much to say about that. But I have read the emails people have been sending about the problems they have encountered and that will be helpful.
ReplyDeleteGood to hear that the data collection went well. Good luck with the coding and don't hesitate to email if you have questions.
DeleteThis comment has been removed by the author.
ReplyDelete1) The data collection process in itself went fine, the pupils were very interested and concentrated on completing the tasks. I had problems finding participants, though, these weeks being hectic for many pupils with exams and other tasks to hand in before the "sportlov". I finally found two pupils who had the time to participate.
ReplyDelete2)It's difficult to know what to do when pupils use completely wrong words AND spell them incorrectly, and at the same time make a grammatical error. I've made some new categories to cope with some very common error types in my material: "missing diacritic", "wrong/missing pronoun" and "wrong auxiliary verb/copula".
3)I'm not sure what to do or how to calculate all the percentages (e.g. in the learner responses sheets). Is it the percentage of the total amount of errors, all categories, or within each category?
I agree that it sometimes is really hard to know what to do with some mistakes, that includes several mistakes. I took your advice and coded them as several mistakes.
DeleteKent: We discussed this problem on Skype - how to calculate the percentages in the excell spreadsheet. Our conclusion was that the percentages have to add up 100 % per informant. In the end we can compare, for instance, how many percentages of informant 1's errors were competence errors compared to informant 2's? I guess it also depends on how you have chosen to differ your subjects. For instance in my study I have 6 informants, two from A1.1, two from A1.2 and two from A.2.1, which makes the results interesting to compare. To add up all the errors to represent 100% makes it hard to then compare between subjects...
DeleteTina: Your conclusion is correct. I think the easiest way of answering this is to give a fictive example. I upoloaded an example analysis (1 subjetc) in the Excel spreadsheet on
Deletehttp://ikt-sprakundervisning.wikispaces.com/Project+2
The file is marked with today's date and "NY!!"
1. Really well.Students were on time and enjoyed doing it.
ReplyDelete2. In the data collection: one of my texts was so full of errors that it is difficult at times to understand what the studente meant. The audio file helps, though. Almost every word in her text has two mistakes to it.
3. So far I have only corrected and coded the texts on paper but not put the eroors into the coding scheme.
I must say, also, that our e-mail thread with comment is working really well for sharing examples. Thanks Henrietta for starting this! I have read many of your examples many times ;-)
DeleteI agree, very helpful to listen to others ...
DeleteAnd thanks to all of you for your responses to my examples! I may not have replied, but I have read your comments and they were very helpful! Also, I read all your examples via email and if I don't answer it just means that someone else has beaten me to it, reponding with the very same ideas...
DeleteGlad to hear that things are going well so far...:)
Delete1. It went well. I had 4 students who voluntered. Had a problem with i microphone, but it was sorted out.
ReplyDelete2. The challenges of data collection I guess for me was to get the technique working.
The analysis takes time! I needed to go through the material several times and also I found it necessary to contact others (Kent, Tina, Linda S, Lisa = the "Spaniards" (sorry Linda ;-))
3. I have already sorted out some.... more will come :-)
Yes, the analysis takes time! It is certainly true. This is why you need to limit your number of subjects/learners. Don't hesitate to email should you have additional questions.
Delete1. I have done one pilot where I was able to troubleshoot for problems for the data collection and analysis. For the real data collection I made sure that the instructions I was going to give the pupils were the same and that they were clear (printed them out and had them on a separate piece of paper), both for the writing and for the use of the spell checker. Not all pupils know how to use the spell checker:) I also tried to ask questions in the stimulated recall about the spelling mistakes that were hard to decide to code as either performance or competence errors. I printed out a copy of the pupils' written text so I could go through the text before the interview while the pupils were using the spell checker. That way I had a chance to prepare for the interview.
ReplyDeleteI have collected data from two pupils and I am going to meet two pupils at 15.00 today who are going to do it together. I want to see how they use the spell checker when they have the chance to collaborate and I intend to audio-record the pupils' interaction.
2. When it comes to the data collection some challenges were technical ones (but doing the pilot helped me out). I downloaded Screencast-o-matic to the computer, made sure that the frame covered everything that was important on the computer screen, recorded the stimulated recall on my iPhone (which I forgot to turn on during the pilot...). Other challenges were organizational ones; writing a letter of information for consent to participate in the investigation took time, also informing my principal and vice principal about the investigation had to be scheduled, setting up the experiment and making sure the same information was provided to all of the pupils. The biggest challenge was/is the stimulated-recall interview; deciding when to stop and ask the question, when to keep quiet to let the pupil have time to think and answer and also allowing for the pupil to comment errors in the text. Pupils are very different, some talk a lot and some don't.
3. When it comes to the analysis I have some questions that I emailed to Maria F. oct Linda B, but I will post them here too:
• Word detects punctuation mistakes, should we categorize and code these as well?
• Word detects the omission of space between a comma (,) and the word after the comma, should we categorize and code these as well?
• If word detects the spelling mistake but only gives one option should it still be coded 1?
• Some grammatical errors: thinked (thought), drived (drove), drinked (drank) seem to be intralingual (overgeneralizing the –ed construction in the past tense), but there is not such category for grammar errors only for spelling errors. What to do?
• If a spelling mistake for example soutch (such) is corrected by the student without using Word, because the correct spelling is not given, but the student fails to spell it right and spells it sutch (such) and then Word detects it and the correct spelling is suggested and the student chooses the right alternative, should this spelling error be coded twice?
Karina this is what I think about your bullets:
Delete1+2. I have created a new category called "punctuation". In this category I have listed all the missing stops, commas, special signs for Spanish. A LOT of these were made! And also I realized that some of the signs were not possible to print using a text editor program for writing. I have coded all the punctuation mistakes, but I might leave these out in the analysis if I don't think they are necessary.
3. I have coded these "1", because even though there's only one suggestions it is still a "list" in my opinion - just a very short one.
4. These are all competence errors according to Rimrott and Heift and I have ONLY coded these as C2, intralingual.
5. I would code this twice, since the student commits two errors albeit with the same word.
This is only my opinion and I am by no means an expert! Hope it helps anyway :-)
Karina: Word detects punctuation mistakes, should we categorize and code these as well?
DeleteJonas: If you like but this can also fall outside a study on spelling and grammar.
• Word detects the omission of space between a comma (,) and the word after the comma, should we categorize and code these as well?
Jonas: same as above.
• If word detects the spelling mistake but only gives one option should it still be coded 1?
Jonas: yes!
• Some grammatical errors: thinked (thought), drived (drove), drinked (drank) seem to be intralingual (overgeneralizing the –ed construction in the past tense), but there is not such category for grammar errors only for spelling errors. What to do?
Jonas: I would code the above examples as spelling errors (Intralingual source). But there might also be grammar errors involved (agreement error).
• If a spelling mistake for example soutch (such) is corrected by the student without using Word, because the correct spelling is not given, but the student fails to spell it right and spells it sutch (such) and then Word detects it and the correct spelling is suggested and the student chooses the right alternative, should this spelling error be coded twice?
Jonas: I would suggest to code "soutch" only one to keep things simpler Suggestion: undetected spelling error (multiple performance error)
Thank you!
Delete"thinked" I would code as a spelling mistake, since it is a "non-existing word" (Heifft/Rimrott), competence error/intralingual.
ReplyDelete?
The data collection process went fine. I have a slight problem with one of my three students. Even though I told him in the instructions to use the help function in Word he sometimes (not always) submits the correct target word without consulting word. One example is "helpfull" where he sees that there is one "l" too many without having to consult Word. He does this because the spelling mistake is so obvious to him when the spell checker is turned on.
ReplyDeleteI also have the same situation as Karina with one student having misspelt a word and not getting the right alternatives in the help function. He then changes the word a little and gets the correct target word in the list and submits it. How to code this?
My informants were happy to write and check, but I had to ask several students to get some that were interested in participating! As I have 9th-graders writing in English, they write quite a lot and make fairly few mistakes. Several of them did not know about the spell check and were really thrilled to use it! I have included the punctuation mistakes and comma-space mistakes that Word recognizes, as the pupils have to react to what Word says...
ReplyDelete1. My data collection had a very large hick-up in that my grammar-check was noneexisting :-( I have tried talking to the IT unit at Lund, but haven't received a reply yet (it's been 72 hours...). My husband tried helping me and it seems that Word 2011 does not come automatically with grammar installed for Spanish - for almost all other languages, but not Spanish... I have collected material with regards to spelling from 6 participants: 2 from A1.1 (year 7), 2 from A1.2 (year 8) and two from A2.1 (year 9). I was going to control for gender aswell but seeing as we only have one boy attending year 9 Spanish and he was not interested the study will have to do without this element. I am thinking about trying to round up the 6 informants again if/when I get the grammar check working, but I don't know...
ReplyDelete2. Grammar check was the biggest challenge. Also the internet was down at my school for one of the days when I had to collect 4 out of the 6 data. I solved this by downloading the screen-cast-o-matic as an app for my computer. This meant that I no longer had to be online to complete the collection. Being a control-freak I also bought a dictaphone so that I could keep track of the sound while recording. I will need this for my own study as well so I saw this as a great opportunity to familarize myself with this new tool.
3. I am a bit worried about the dead-line for the essay, since our course in Göteborg is starting the same week and I am asuming there will be lots to read. My goal is to have the essay finished at the same time as the presentations, but I don't know if this will be possible. I also find it hard to start writing the essay, what to include in the introduction and so on, but that might be just me and my style to worry about absolutely everything :-)
It was a pitty about the grammar checker. I didn't know about this. But don't worry about it too much. You will have a lot of data only by looking at spelling errors (since you have so many participants).
DeleteAbout the paper: we can talk about this next Friday. In the meanwhile I suggest you start writing on the method section and the analysis.
ReplyDelete1 - Everything went fine. The pupils enjoyed doing it, it gave them the feeling of doing something important and they liked it!! :-) No technical problem.
2 - Like every "first time" is this process really challenging but rewarding! To have the possibility to ask others in our group helps a lot. I have written down all results in page 6 (coding) and I'm working with the others now. I received a lot of answers today and will certainly have a lot of new questions next week... :-)
Good to hear! You are welcome with your questions!
Delete