We are all aware of the economic challenges that schools face. The truth is, schools have always had limited budgets and been scrutinized for spending decisions. In the wonderful age of accountability, those decisions seem to be under a much bigger microscope and lots of questions get asked. That’s not necessarily a bad thing but it certainly requires us to think about how we justify spending.
Professional learning comes in all shapes and sizes and I’m hard pressed to assign value to any. Those who speakabout “training” usually are speaking about a linear, by the book approach which is often easily transferred into the classroom. These are not transformative types of experiences but sometimes necessary to help teachers do their job. The professional learning that moves to make us truly better teachers and provide the best learning experiences possible are the really expensive ones. By expensive, I’m not referring exclusively to money but largely time. If change were that easy, we’d all be doing it. The hard work of improving practice generally takes years and as we all know, teaching is a profession you never quite get right and the very best never stop learning. By that standard, professional learning is simply what we do. You might say, teachers are professional learners.
A rough calculation conducted by the Business 2 Community bitcoin casino sites would suggest that in the last 10 years, my school district has spent upwards of $35,000 on my professional learning that includes conferences, classes, and workshops. That’s a lot of money and I realize that it’s probably more than most teachers would receive. (Although most of that has no substitute teacher costs which for most teachers, represents a significant expenditure) Was it worth it? There’s no doubt that some of those workshops and conferences were less helpful than others but the whole of those experiences I believe are greater than the sum of the parts. Out of all the conferences I’ve attended, I’d be hard pressed to recall a single event or experience that has significantly shaped my work above any other. However, I can’t imagine being anywhere near the educator and leader I am today had it not been for those opportunities. I also recognize that not all professional learning costs money. This space has been one of the greatest sources of professional learning for over five years. The time invested here is significant but has been worth every moment.
In the effort to be fiscally responsible and focus our attention on what truly matters, we hear this question asked almost everyday as we make choices on how to support our school division: “Will it improve student learning?” That’s a simple and yet very complicated question. It’s pretty easy to suggest that almost anything you do will improve student learning but not everything will translate into the classroom and impact student learning immediately or directly.
Here’s what I want to know:
1. How do you measure the effectiveness of professional learning?
2. Is there a time frame in which the professional learning translates into student learning?
I’m not sure I’m articulating myself particularly well here but am searching for some ideas that will help me solidify my own beliefs. I’m really curious to hear your thoughts on this one. Thanks in advance.
The obvious answer is the effectiveness has to be measured first on how well the PD met the objectives it set out to accomplish. Having clear cut objectives – intentional direction – is a hallmark of PD that results in prof. learning. If the goal of the PD is to teach teachers how to be more effective math teachers, what are the specific objectives that will help them become more effective? It can’t be all theoretical, compelling case for improving hype. There needs to be specifics in terms of measurable change.
First you set the goal. Then you set the specific measurable objectives. Then you actually follow-up in ways that are not just self reported data (maybe classroom observation, focus group with students, looking at student work, etc.) For example, if an objective was: all learners, as a result of the PD, will be able to teach specific tested concepts maybe… quadratic equations by using a 21st Century strategy –let’s say the TPACK method.
Then there needs to be both formative and summative evaluation for the learners (teachers) who participated in the PD. So the first thing you do is embed lots of process pieces that result in created artifacts that show whether teachers mastered the concepts being presented. Mastery needs to take place at all levels of Bloom,s or some other cognitive measure. Are there examples of mastery through knowledge, comprehension, synthesis or analysis? Can they evaluate their own product in terms of the objective-how about others work? You provide places for them to metacognate about what they are learning and to hear each other explain what the learning means. Collaboratively you look at examples of best practice and then they create something that will show they mastered the concept. You ask lots of questions, have your learners do most of the talking, and generalize to authentic examples.
Here is where you measure for student learning and it occurs immediately–3 months–6 months– 1 year out.
After the PD is over you visit classrooms to see how the new professional learning is going. You use observation check lists that look for specific use in relations to your intended objectives. You look for scale- how are the concepts being used in generative ways you hadn’t considered in your PD. You talk to kids and ask, “What are you doing? What are you suppose to be learning? Explain to me how to work that equation. What is TPACK?” “What can you do differently now that you couldn’t do before in terms of your own personal learning and understanding of this concept?” You record the answers so you can share with others in the school and in the blogosphere. You talk to the teacher and ask them to share what’s working from the PD, what’s not, lessons learned. You ask them, “What could I have done differently in my PD offering to have helped you grasp this concept better?” “What did I do that really worked?” “What do you need from me now?” “What professional learning goal can I help you achieve- one you are interested in achieving, not one I mandate, that is related to this concept?”
The effectiveness of PD, the kind that results in professional learning- is dependent on more than the teacher getting it when you are presenting the content. It starts with the PD provider’s planning and ends with follow-through in the classroom. Ongoing authentic assessment of how well the PD was delivered and then follow through to help the individual teacher with what they need to be successful or even what they are passionate about learning as an extension of the PD is what makes PD effective.
The less obvious is that the effectiveness of professional learning is not always measurable. Value add is a tough thing to measure empirically. For example, I was told a story yesterday by a teacher who said that for years she had been trying to get her principal and teachers to buy into the reasons and ways to bring about 21st Century reform and transformation in to her school. She felt she was banging her head against the wall. She said after her principal had heard me speak about learning communities as a PD means- something clicked. She finally got it and has been implementing sound 21st Century change since.
The point I am trying to make isn’t oh look I am great. Rather, that keynote was delivered over a year ago and I left having no idea of how effective I really was. I mean I had my objectives for the audience. I knew what I wanted to convey and what I wanted them to walk away with and how I was going to embed process into my keynote to make that happen, but I never knew the value add I had given that principal. It wasn’t measurable then. Make sense?
One of the reasons I love what we do together in PLP is that we are developing PD in an ongoing, job embedded capacity. Remember, it isn’t just about the learners– it is about you as a learner. And not everything that is of value can be measured.
Hope this helps. I am very interested in what others have to say.
Thanks Sheryl for that thoughtful, well written response.
Part of the problem I see both with measuring PD as well as student learning is that quality measures and assessment take a great deal of time and money. Meaning to effectively follow up and support is more than having a teacher complete a checklist. The deep learning that PLP centers on is not easy stuff. I’ve witnessed some teachers who immediately respond and actively begin to make changes in their classroom. Sometimes for the better and other times with some failure which isn’t always a bad thing during a time of change. Other teachers may take longer to fully embrace the change and there may not be much evidence in student learning for some time. Just like reading and many other skills, not everyone learns at the same time.
The tracking and monitoring of change seems to be the challenge right now. Finding the right measures and the right time and ways to measure are my greatest challenges.
Dean, I think the problem of measuring the effectiveness of professional learning is similar to the measuring of student achievement – in some ways. If we really wish to know what students have learned, we need to look at transfer and application – not merely at the ‘skills’ acquired. With teachers, we don’t typically use tests or rubrics to evaluate the effectiveness of their learning. We look at how they adopt and adapt the knowledge, skills and approaches that were the focus of the professional learning event. Do we often examine this with a formal process? I think not. Is that necessary? Perhaps not.
Another thought that comes to mind is in regards to the nature of the professional learning we wish teachers to achieve. There is a great distinction between learning and applying methodologies/techniques and the transformative learning that shifts perspective and belief structures. For example, with a huge focus on Project Based Learning (pbl), we are seeing many teachers ‘do’ pbl. The ‘doing’ of pbl in the classroom is easy compared to the messy and complex realities of teachers implementing effective inquiry-based, student-driven, pbl. The former may not necessitate that the teacher shift their notions of ‘power’ in the classroom. The latter often requires a change in deeply engrained beliefs that are not just cognitive shifts of methods, but are tied up with our affective sides and our beliefs about life in more general terms.
Ok. No real answers to your questions here! LOL But, some thoughts nevertheless.
If interested check out PBL: The New Worksheet at http://theconstructionzone.wordpress.com/2009/12/01/pbl-the-new-worksheet/
Peter
Thanks Peter,
I agree with your analogies as well with the ambivalence about the desire to measure everything. I think this is part of the dilemma. As I said, I have a very difficult time associated a specific professional learning experience with a change in practice. They all mesh together. Trying to compartmentalized learning continues to plague us in education.
Before I get to answer your question — I do need to wonder if “$35,000” was spent on sending your elsewhere to then return to your campus? Or was it spent on in-house training and PD?? Because I think that is has to be a part of the judging the effectiveness of the PD.
You ask:
1. How do you measure the effectiveness of professional learning?
There are 2 things to take into consideration.
#1 — Has it created new ideas in the learner that will make the learner a better educator? Has it transformed ideas, solidified ideas? Did the PD make a difference – whether dramatically or simply? Was there change?
#2 — Did the Professional Learning ripple out to others on the campus or only made an impact on 1 person?
But then you have to wonder — as Sheryl said — what was the main objective? If the main objective is a singular purpose…..should the district be paying for it? Hmmm not sure on this one.
2. Is there a time frame in which the professional learning translates into student learning?
I wish I could strongly say “YES, immediately”….but I cannot.
I think Professional Learning is like reading the Bible. Sometimes scripture does not make an impact and then at other times, it becomes crystal clear what was said and what needs to be done because of the realization.
There are MANY sessions I have gone to that took a while to make an impact on me…..and my students.
Others, the change was immediate.
If we are all learners — some learning takes time and to expect instantaneous change (that rolls over to the classroom) at every single professional learning event — I think that would be doubtful.
So, back to my first thought.
If your district is paying $35,000 for you to learn…..away from campus…..I would then say that there has to be a HIGH EXPECTATION that when you come back, you MUST share what you learned (either verbally or by example).
If your district is paying $35,000 for your campus to learn — as a group — with common goals, common focus, and a shared vision…..I think that would be easier to truly measure the effectiveness.
Just my thoughts
Thanks for letting me share.
Jen
Jen,
My calculations are a mix of on and off. I’m not sure it really does matter. While I agree with you about expectations to share, I see this as being a pretty vague concept.
I remember the very first conference I attended and the application that I had to fill out at the district level had a check box that asked, “Would you be willing to share your learning with staff?” I checked the box but was never asked about the conference ever. I think in general, there are very few specific expectations. We hope and trust that learning will be shared informally and while that may be in some ways the best approach, most districts have little or nothing in place to insure that ideas spread.
Dean,
I am wondering- just because that is the way it is done… “I think in general, there are very few specific expectations. We hope and trust that learning will be shared informally and while that may be in some ways the best approach, most districts have little or nothing in place to insure that ideas spread.” Does that mean we should settle for that? Or in our efforts to transform schools into places where the focus is on learning and then sharing and reflecting on what we learn – should we just accept the fact PD is what it is and can’t be measured effectively?
Sheryl,
In your response you talked about follow up. I would suggest that follow-up and monitoring in general is the thing we do most poorly when it comes to PD. I think there are about 3 reasons we don’t do a good job of follow-up and support.
1.We assume that “we taught it, so they should know it”. The same problem with some teachers. We trust that as professionals, they should be able to figure stuff out on their own. Being a professional means you should be up to the task.
2. Follow up and support is costly: important but costly. If you’re going to ask others to take the time to observe, monitor and support teachers, you have to be prepared to provide the time and resources for this to happen. This obviously recognizes that my first reason isn’t valid but at the same time, I’ve not seen this done particularly well.
3. Support and monitoring is seen as evaluation of teacher effectiveness. Lots of teachers don’t want to be monitored and lots of those doing the monitoring don’t do so in the spirit of support but as evaluation. This is about trust but also culture.
All the above barriers are significant to overcome. I’m not suggesting they are worth it but again, I’ve not seen enough examples system wide that are removed these on a large scale.
In many districts there is such a low expectation for PD. Often the standard of success is “I wasn’t bored”.
Is the time I spend reading this blog and your twitter posts part of my job? I haven’t figure out if this is me avoiding work. Or is reading blogs, posting comments, etc the actual work I should be doing.
I agree with a comment above that too often we don’t know the impact of a conference/talk/session until much later. The instant feedback and evaluation might not be accurate.
I would love if once per week we sent a faculty member to a neighboring school district and they reported back to us the next week on a simple form (what is similar, whats different, what worked, etc…). That would be awesome.
You are very lucky to have that support. I imagine you get to choose much of your PD as well.
Brandt – I love, love, love what you wrote about. I encourage you to write a blog post saying just that. word for word.
And it extends my own point that we now have the tools and the ability to create our own customized PD (via Twitter etc). All we need are the structures and supports (i.e., time and, perhaps, reporting tools) that would make these activities an official and valued part of our present job descriptions (i.e., in the form of customized PD).
Those of us who are working overtime to explore the real world around us do so because it’s a pleasure and a passion. We make the time. But this activity really should be supported and compensated if it improves our teaching and makes our classroom experiences more meaningful for our students. And that is essentially what some apparently tech phobic colleagues are arguing – we’ll do this as long as we’re paid.
Thanks Melanie and Brandt,
I was hesitant to go too deeply into online learning because while I agree there is huge benefits, both from a time and cost saving standpoint as well as customization, it’s not a magic bullet for change and is but one way to improve.
I believe there is still value in structured, paid professional learning especially if it’s tied to sustainable, customized online learning as well. Most districts that are shelling out money are never going to measure the effectiveness of online personal learning because they have no investment. Even if they acknowledge its benefits, they have no obligation to see if it’s effective. At least that’s how I assume most districts would feel.
I feel the learning is effective when it makes me think. Being reflective in my teaching is such an important practice. It is where I make decisions that really effect my teaching and my students learning.
If what I have learned does cause a change in my teaching practice, I would say it is used as soon as I am in the classroom or in the specific situation it applies.
Most of the time my professional learning doesn’t through my district either.I have learned over the last few years that effective learning can take place for me online. I rarely go to workshops outside of the mandated district ones (many reasons for that including they don’t offer me much) and yet I have changed my teaching incredibly. In fact I have only gone to one conference in the last five years yet I have completely changed my teaching beliefs.
That being said, I do plan to go to a couple conferences this year to share what I know and to develop face to face relationships I have build online.
William,
You likely know that you’re an exception in terms of learning online. Does your district have a way to connect PD to student learning or is it like most districts where it’s pretty sketchy?
Dean, student learning seems to take a back seat to getting better scores on the high stakes test. Last year the district in-service was Fred Jones and not once did he mention learning. I don’t think most of the teachers in my district have even thought about their teaching practices critically. They just continue to teach the way they were taught.
Pingback: Tweets that mention The cost of Professional Learning «Ideas and Thoughts from an EdTech -- Topsy.com
Hi Dean,
The literature on the scholarship of teaching seems to point to the same things repeatedly and can basically be laid down to taking a research perspective on it, making the process iterative, and then sharing your findings with as many colleagues as possible to feed into your own reflective practices to help you design modifications into your next teaching plan.
Analytics may be part of the design of the research, however I’m starting to believe that measuring ‘teaching’ is to ignore the wide palette a good teacher employs in their profession. How do you measure a students engagement or enthusiasm for the subject being taught? How do you measure the creativity a teacher brings in presenting their topic? Why do we focus on standardising the results of individuals (both teachers and students) who come from such a wide base? Doesn’t this focus skew our perspectives and concentrate our attention away from other areas that might be important too? Lets trust the resulting paper, presentation, blog will speak for itself, and to other teachers.
And lets help teachers see themselves as scholars, surely learning is best facilitated through action. Surely the life-long learning skills we need to instill in our pupils needs to be role-modeled by us.
Andrew,
The balance between the value of measurement and misusing it is something we as educators face everyday. Only thoughtful educators wrestle with this. Those that don’t are, in my opinion, being dishonest.
The idea of life long learning is a great example of something that can’t be measurement for a long time. Perhaps our best measures would be to see if our students were implementing the values and ideas taught to them in school 10 years after they graduate.
Well,in the end we can only hope for that.
If we can only see what the students are doing 10 years from now.