Let's talk about tests, baby

Still considering the notion of assessments, both formative and summative, in the world of online classes. I think it meet to start assembling a list of pros and cons, limitations and opportunities. Maybe that will help me devise some method of testing which I would see as honest, fair, useful and full of integrity. You know, educationally sound.

First -- memorization is out as a skill. I can't test it. I can't monitor students in real time to be sure that they aren't relying on some other resources. I know everyone loves to promote higher order thinking skills but lower order thinking skills are still vital and are much more difficult to test virtually.
Second -- synchronous testing (much like synchronous classes, according to many experts) is difficult and probably a waste of time. Getting everyone the test and having all be able to access it in the same way at the same time, and take it and submit it on the same schedule seems daunting (ignoring the issue of the administration of that test and the possibility of cheating)
Third -- many would say to co-opt the strengths of online learning by allowing students to collaborate and create. This is nice except it invokes questions of personal responsibility for a grade (and monitoring participation in group work and allocating fair grades, which is already, even in person, a problem) and, in my humble opinion, it ignores a truth that the future will not be all about collaboration. Even in the workplace of the future, individuals will have to be able to perform on their own.

Exit tickets -- they aren't always relevant. If we are doing Kahoot presentations by 4 students per day, what does another student have to email me? A question that someone used? Who remembers that? Also, any ticket will only show that a student was paying attention at one moment. And how many times do I have to repeat announcements and assignments and students still don't get them. Then they ask "can you post that to Haiku" or whatever other HW platform we use. So simply saying, with 3 minutes to go "before you leave, send me an email with ______" is doomed. And, best case, it means I have to plow through 75 emails to see if they are "good" -- what is the standard for successfully completing this? Is it "done or not done" with the grade being a pass/fail?


There are three modes I am starting with for assessing and I have jotted some notes under each one discussing the problems and limitations associated with it:

1. Timed, written responses (short answer, multiple choice or essay/paragraph)

1a. Enforcing a timing is possible using a third party product that opens and closes the assessment on a fixed schedule, but because of the possibility of "extra time" separate assessments would have to be made for each student to accommodate needs.

1b. Even in a timed situation, it is impossible, remotely, to ensure test integrity

1c. Responses are all written and have to be submitted electronically (typing is a skill not every student has or is comfortable with). Students who struggle with written communication will be disadvantaged by the demanded form of response.

1d. The long form of this (the "paper") stays valid but is not complemented by other assessments which address aspects that a paper cannot account for.

2. Responses to direct questioning (which assesses both knowledge and attention)

2a. Not every student is comfortable answering questions on the spot, participating in conversation, formulating spoken word answers. To shift demands and require that students become something different from what they are (in a classroom that has tried to value who they are now instead of demanding such change) undermines the teacher-student trust-relationship and asks for something unfairly.

2b. Not every student can pay attention via computer in the way that we want to ensure. The distractions and demands are very different. We cannot command, nor can we expect to command, the kind of focus that having students in our classroom allows for.

3. Presentation (individual or group) - a recap of knowledge or a demonstration of ability

3a. Ignoring the problems of group work interpersonally, the technological demands present a problem for some students.

3b. The presentations (technologically) are limited and rarely seamless, wasting time. Even in best cases the time it takes to go through presentations of any sort eats up class time.

3c. It is impossible to monitor both the content and other students in the class unless another level of assessment is added in which then has to be monitored/graded.

Tomorrow, I'm planning to throw this quandary to my 12th graders and solicit insights, maybe hear what other teachers have done, or see what students would respect. I'm also asking other professionals and posting messages on the relevant web forums. More updates as events warrant.

Practical Magic

So I am still intent on making this work, this whole teaching from home thing. I'm still running into issues (and I am psychologically dismayed that my homespace now requires that I adopt my work persona...I had hoped that never the twain shall meet) which I will catalog but I also want to keep track of what I am doing (other than feeling nauseated from looking at a screen for so long)

First, problems:

I still am using a single laptop for my Zoom meetings. I don't have 2 screens and I don't intend to install a second screen so that I can do what I should be able to do without any computer -- teach a class. That would be TOO much technology. I try not to have to use my phone at the same time, but I know that one of my teaching tools requires that students use their computers and their phones at the same time. This poses its own problems.

My internet connection is being strained. With me on Zoom (and other things), my wife on Zoom or the internet, my kids streaming TV shows (and sometimes, their school lectures) I am really pushing it and I get the messages on my computer telling me that this is a problem. My connection lags, and trying to RDP into work to get those files while also working on my own connection is not working so well. I know of students who have no camera on their computers, or their cameras are broken. This is not a made up excuse -- I know it to be true. Not everyone can afford a shareable high-speed internet connection. Some students need to turn off video to save bandwidth. Keeping their attention is getting tougher and tougher and they are getting better at setting up convincing virtual backgrounds to fool me into thinking that they are there and paying attention. This requires more energy from me, verbally. I can't walk around and use my body language to fill the gaps -- silence means nothing is happening. That demands a lot more from me, to fill the gaps.

I have my 12th grade class presenting videos and memes that they have found with certain topics -- this class is on humor so our sources are multifarious and all viable. They are 12th graders, though, and are already somewhat "checked out" of the high school experience. Those who are engaged normally are doing a great job. Some who are quiet continue to be, and many are just not in the game. They have too much fun with cross-conversations and I spend too much time chasing noise until I mute them all and show a video, with no way of knowing if they care at all.

My 10th grade is setting up Kahoots, short quizzes that they make and share with the others. I can't "grade" these quizzes as I would my own, so I need to keep track of who has done it and give credit for that. We (in order to be fair and ensure that all did the work) have to see each person's kahoot -- in a class of 25 that can take many class-hours and an be tedious and/or boring. The technology isn't so seamless so students still have to learn how to share screens, balance 2 screens (on to host or see the questions, and a phone to answer) recognize limitations of hardware (what you can do on a phone, you can't do on a computer and vice versa) and resolve error messages (all assuming the internet infrastructure in their neighborhood and home holds steady). They haven't gotten burned out on it yet, so we will keep on reviewing them and then getting to actual discussion about our text. But simply introducing new third party apps everyday requires that I achieve mastery quickly, and that I find a reason to use these in a pedagogically organic and responsible way and for the most part, I haven't.

This brings up a point about assessments. I have to replace my timed, structure summative assessments (in class essays, short answer timed reading quizzes, vocab quizzes based on memorization) with more frequent assignments. I wouldn't call these formative because they aren't being used during instruction to help build knowledge. My structure is still "teach, then test" and if that has to change to accommodate this new reality, then so be it, but I need to be shown how so I can build a new curriculum with a new paradigm in mind. These short assignments need to be read, responded to and "graded" (whatever that means). This takes more hours than my traditional modes of assessment, and increased assignments compounds this. The spontaneous "pulling a kid aside and pointing something out quickly and moving on" has to be replaced with a formal email, detailing in writing, not speech, what needs to be addressed. Exit tickets also require my review -- and what counts for a "good" vs. just a "done" assignment? The students are craving grades, asking if and how I will be grading anything they do. This brings up a question of student expectation.

I can learn to change. it takes me time, but, for myself, I can change how I do things and such. But asking students to change from an educational structure which has defined them for 10 years is difficult. They don't see a big picture -- they see their scores, grades and standard methods of measuring their success. And I don't just mean for the sake of college applications -- they have internalized the lesson we have been trying to get across to them -- that our grading can be used as a reflective tool to help them see how well they are advancing through the educational ranks. And now we have to ditch these measures and reassure them that they are getting smarter without some external device like a test? That demands more of them than of us!

Some students have handed things in late, claiming technical issues or emotional ones (a student is quarantined away from her family and is having a very hard time). Attendance is still almost impossible to take. With students shifting position on the screen, entering and exiting and not always having (being able to have) cameras running, it seems a waste of energy.

I am going to have to change the parameters for my research project, recognizing that students have not already, and will not be able to get to their local library. I will have to make all resources required digital (which I find abhorrent as students still don't have a sense of what online is trustworthy and what is not, nor do they understand that not everything is available in digital form). This will reinforce their false notion that online research is good enough, and this makes me sad. The hard part has always been convincing teenagers (who, clearly, already know everything) that there is more to find than just websites (and they don't know what a "website" is as opposed to anything else -- it isn't a discrete entity/object, so they don't know that they can assess it on its own merits.

I am using the AP Classroom to create short quizzes and assign them to the AP students but the test bank is limited, the explanations are scant and the students can't seem to care as much as if there is face to face accountability. That continues to be a problem -- they just don't care as much and would rather use class time to have any social interaction rather than be forced to focus on academics.

I try not to get frustrated with all of them, their technical naivete, their disinterest in schooling. I try to deal with having to sit and stare at a screen for hours on end, reading emails as homework instead of working with a pen and paper. I try to wrap my head around the concept of grading and testing as they (don't) apply to this new system. I end up really tired.

Notes from the undergrounded

I am still learning this whole video teaching thing and in some ways, it can do the job, but there are other areas in which it is deficient. I have seen contradictory information about scheduling learning times vs. allowing students to connect at their own pace, about assigning fixed work and lots of it vs. assigning significantly less and very different work and other basic educational (and paradigmatic) binaries. I don't know where I stand or where others will tell me to stand, but I want to keep track of my learning curve on this, so bear with me. You are in quarantine. What other choice do you have?

We are using Zoom and it is not horrible. But, and I have tried to pass these concerns along, there are glitches.

Sharing screens and limiting who can annotate is tough. The white board app initially refused to appear when I said that only I could write on it, and for other annotation tools, I believe that permissions are all or nothing.

People still get muted for no reason and unmuting them doesn't always work. I, on the whole, prefer to keep students unmuted -- the ability for them to interject, or add something without having to raise a hand and wait to be called on more closely mirrors how I run my classroom. Unfortunately, incidental noise can hurt our ability to hear other people who are talking. One student shifting his computer on his desk, or putting his phone down on his bed can take over a room and it isn't always easy to spot the source of noise and mute selectively.

Attendance is spotty at best -- students come in and out, so keeping track of them after initial attendance has been taken is very difficult.

On my screen (and I only have 1 computer to work on) I can't see 49 people on gallery view. That option is greyed out for me. I see up so 20 or so in gallery view -- sometimes when I have over 16 people present, I only see those 16 (with no "second page"). And because students keep moving in and out, and talking moves a student to the forefront, I lose track of students.

Students are supposed to have their cameras (and microphones) on -- unless I mute them. Here are some excuses which I have heard:

a. I am on a desktop so I have no camera installed
b. my camera (my software) is broken
c. I am in a room with others and it is noisy so I am staying muted
d. My room is dark so you can't see anything


Keeping students attentive and or focused in really tough. We are encouraging them to have access to other screens so they often are fixated on those other screens. Even if they have the video room open on a single, primary screen, they can have it minimized and be doing other things on their primary screen and yet appear to be paying attention. I have students who access on their phones and then say that they can't get chat messages, or open linked websites while maintaining presence in the video component. One had to exit and start up (and join via) a computer. This adds to instructional time. It is also harder to make sure students are progressing through assigned work. I have more students do the whole "wait...what are we doing now?" thing repeatedly, or ask questions just answered because they weren't fully paying attention or the audio didn't make someone else's question easy to hear. [if any teacher gets fooled by a virtual background picture/gif of a student, then that teacher deserves no mercy, but the students are trying to fool us and with people in and out, they can fool us because we can't check consistently]

There is pressure to use outside, third party add-ins. These are not resources that I use on a day-to-day basis when I teach so jumping in to the video-mode AND having to integrate unfamiliar technologies is doubly daunting. As a note, understand that I don't use those other options because I am a technophobe, but because they don't arise organically in my teaching method. This is asking for major shifts in how I approach the classroom. As a parallel, students do not seem comfortable with chat boxes alongside video teaching. I tried adding in a third-party component today and the students found it glitchy and hard to work and incorporate into the classroom. We will keep trying tomorrow and see if this can flip the source of content to students and if it can change the nature of assessments.

I am trying to keep my video room open as a drop in center, but to do so means keeping it in the background (with my camera off so that no one accidentally sees my reaction to stupid memes). But the room closes automatically after 40 minutes of inactivity.

This is not homeschooling. In homeschooling, a student doesn't wander into the wrong room.

Long Distance Runaround

Note -- this is not a screed; it is an exploration of ideas, challenges and changes so that I can work through them and be better than I am at what it is I have to do.

I am in a state of aloneliness. That's not a bad thing, and often is my desired status but now, it is being requested of me in response to the outbreak of fear concerning the transmission of the coronavirus. So I'm staying at home as socially distant. I have been emotionally distant for years, but now, this.

I'm OK being stuck at home. That's where much of my stuff is. But I also recognize my responsibility to my students. Not to brag, but I'm a teacher so I'm supposed to be telling 15 year-olds stuff that they don't already know, and didn't know they are supposed to care about. So the school has sat us all down (virtually) and given us a crash course in setting up video classrooms. They are much like regular classrooms except totally different. Now I know how to set up meeting times, share my screen, and talk at my computer. I'm a expert!

Now here's the thing -- I have been teaching on the high school level (that does not refer to my teaching, but to the people I teach) for over 25 years. I know what I'm doing. That doesn't mean that I'm doing anything right, well, or competently, but I know what I'm doing. When I screw up, I know it. When I succeed, I'll know it. I don't believe that my skill at teaching a live class of between 5 and 50 humans can be transferred and replicated virtually.

Part of teaching is the individual nature of instruction even in a packed class. Being able to walk through a group and assess who is on task and who isn't (and, often, why and whether it is worth calling out) is part of the job. If I have, on my screen, the video of the 4 people who spoke most recently and maybe also 4 people whom I choose to watch all the time, I will be unable to watch anyone else. If some child is quiet and isn't one I have to worry about actively, I might never see him through the class and have no idea if he is clicking with the material.

There also seems to be no particular way for students to "raise hands". If I am lecturing (another problem -- in a world of flipped classrooms, this delivery system seems to revolve around the facilitator of the classroom making lecture the easiest method) and a student has a question, I will either get a notification if the student speaks (which is an interruption), or puts something in a chat box, which requires that I process a different set of visual inputs while trying to watch the various students. The sense ratio will be very different. I also fear the student who, assuming I didn't see a comment in a chat box, floods the box with additional requests, so I have to remain vigilant and address each chat statement verbally (instead of through eye contact or a wave of the hand, indicating "I see you, but I'm not ready for questions"). The differences in platforms and options available to each (web, phone, app etc) should also be considered as I have already run into situations where what others can do I cannot because I am using a different interface.

And what about the content? Do I prepare a video for the students to watch in advance of or during the class meeting session? I have always resisted preparing videos because they are too unidirectional and non-spontaneous. I know what some of you are thinking -- why should the class be spontaneous? Don't you have a curriculum and haven't you planned out what you are going to teach? Yes and no. I teach, this year, four sections of the 10th grade, all ostensibly on the same tracked-level. Let's say I want to begin a unit on poetry. I need to assess prior knowledge by asking prompting questions, and I have to watch, as I present information to see what is not sinking in so I can either represent it, or move in a totally different direction, salvage the balance of the class-time by presenting other material and get back to a difficult topic on another day. I might realize that students who claim to have prior knowledge really don't, or don't have what I need them to have, so I have to change gears and cover earlier material again. But I only know this on the fly. It can't be prepped for -- a teacher has to think on his feet, something hard to do while sitting down on one's tush.

The nature of assessments has to change as well. Timed, summative evaluations are not impossible but are tough -- they have to be set up through various third-party services with an "assessment open" and "assessment closed" time, but even then, students are on their honor not to consult each other, the internet or other resources to do the work. It does beg the entire goal and utility of timed, summative assessments, but I still see them as valuable and, though I generally trust most of my students, it isn't proper to put a stumbling block out there. If one student cuts corners and has an improved grade, others will be disadvantaged. So I can do more formative assessments I guess. But those require a lot of tech set up as well, and their nature is so static. I already do many of these when I ask for participation or groups to present short ideas but again, some of the spontaneity is lost here. And if I tell the students to work on a long term project while I sit at my computer, ready to interact with individuals who have questions, is that really a virtual classroom? I do this in class sometimes, but on the computer, a private question by one student can be heard by all and unless I have set up separate chat avenues for each student, and monitor them, what one types, all see. This might not be embarrassing, but it is disruptive.

Also, can I sustain anything like this for a formal 40 minute period? Monitoring who is there, who is paying attention, who is getting the material and still presenting content is tough when done through the computer. Forty minutes, live is tiring but I have developed an approach which allows me to do this. It took many years of watching, practicing and studying before I got even OK at it. Now, we have to change the entire approach, and possibly the aim of the class and the standards of evaluation and get it right from day one.

Note -- I am not even addressing the changes in demands and expectation on the students as learners. First, you have the technological (and financial) demand. While in this community it is presumed, there are still students whose families do not have (or approve of) extra technology. Then we have to consider that everything from behavior to participation to performance will change, and the one thing we have been worried about (that they will be distracted by screens and technology) will now become the sine qua non of instruction. Talk about a stumbling block. We are all but demanding multi-tasking, which never quite works, and expecting the kind of behavior and awareness which we cultivate face to face, when there is no way to enforce anything. Say a student moves his computer so he is off camera. Am I supposed to say something? Or he is on camera and staring intently at the screen. How do I know what he is really doing? Or he puts his microphone on mute (so as not to disturb class discussion) and then puts his phone next to him and makes a phone call on speakerphone, being totally distracted while apparently listening intently. A student can make all sorts of excuses (my wifi wasn't working, my computer is slow, the software didn't work for me etc) and I can't question (or remediate) any of them [and what happens if MY technology isn't working perfectly?]. Group work done virtually compounds this as students have to monitor each other (not an option in our school software -- they will have to be working on multiple screens which introduces a whole exponential host of problems).

This will all demand extensive cultural, psychological and pedagogical change. None of that change is impossible (though it is tougher for some of us ol' dawgs) and I think it will require rethinking what we hope to accomplish through education. If it succeeds then we will have to ask the reverse question -- if it can be shown that goals are reached through virtual learning, then why maintain traditional brick and mortar schools at all? If we think that there is something which virtual learning cannot accomplish, then we have to be ready to embrace that limitation now, instead of assuming that we can do a land-office job from day one, and ignoring (and anticipating) the problems. While I know that we need a solution NOW and we don't have the option of implementing slowly, testing and revising bit-by-bit, I believe that we need to validate concerns and work through them as real challenges instead of brushing them aside and reassuring ourselves that "we will figure it out."