New Education Reporting Requirements: Introduction and Partner Q&A

New Education Reporting Requirements: Introduction and Partner Q&A

December 9, 2019 0 By Ronny Jaskolski


Okay, I think we’re going to go ahead and
get started. Welcome, everyone. Thanks for joining us
today. You are in the USAID Education Sector Reporting webinar and this is
just really a chance for you to get some updates on the new reporting
requirements and then to have an extended Q&A with some of our field
experts. So I’ll introduce our presenters in just a moment but first I just want
to orient everyone to the Zoom platform. If you haven’t been on Zoom yet, welcome.
Just a few key features of the platform I want to orient you to. So we have our
unmute and mute function over here in the bottom left corner of the screen.
Everyone is automatically muted when you come into the room so if you’d like to
come off of mute to ask a question later in today’s session, you simply click on
that microphone icon on the left corner of the bottom of your screen. You’ll have
to roll your mouse over to see that toolbar appear and you can click on the
unmute button. If you have trouble with that, you can also type your question
into chat or ask for help and I can unmute you as well. To find the chat pod
you also scroll your mouse over the bottom of your screen. You’ll see a
toolbar appear and a little thought bubble icon with the word chat. Go ahead
and open that. That will be our chat function for today and we’ll be queuing
up some of our questions for experts there in the chat box. So encourage you
to use that. I’ll just let you know that you can change who you’re sending a chat
to by using the drop-down menu next to the to field. It should be automatically
set to everyone if you type a chat there, everyone will be able to see it. But you
can also select to type a chat to one individual so that option is available
for you as well. So with that I will invite our presenters today to do a
quick introduction. So Chris would you like to come off of mute and just say a
quick hello? Yes, thanks Laura. Hello everybody. Good morning. My name is Christopher Ying and I’m a senior education data specialist in the USAID
Office of Education. Awesome, thanks Chris. And Rebecca, would you like to do an
introduction? Sure, thank you, Laura. My name is Rebecca Schultz and I’m a
technical manager on the education team at Management Systems International and
I’m supporting Chris in this work through the data and evidence for
education programs activity. Awesome, thanks so much. So we’d like to get a sense of who is in the room with us today so go ahead and type into the chat box your name and
where you’re calling from. the organization that you’re with, and your
role there. We would appreciate that information. We’d also like to get a
sense of everyone’s familiarity thus far with these requirements so I’m going to
open a poll now. You’ll see that poll appear on your screen and it’s just
simply how familiar are you with the new education sector reporting requirements.
So go ahead and give that poll an answer and just give us a sense of the
experience in the room today. Great, thanks so much for those
introductions in chat. We really appreciate that. Awesome and I’ll give just about 10 more
seconds to complete that poll just letting us know how familiar you are
with these new requirements. So it looks like we have a range of not
at all familiar to somewhat familiar so a little bit of experience in the room
and hopefully following today’s session you will feel more familiar with these
requirements. So thanks so much for that. And with that I’ll hand it over to Chris
to share his presentation. Great, thanks, Laura. So once again, thanks everybody for
joining us this morning. I’m very pleased on behalf of the Office of Education to
be presenting to all of you this new education sector reporting guidance. This guidance reflects more than a
year’s worth of work on behalf of members of the Office of Education here at USAID, the work of Rebecca and her team at the DEEP activity and in
consultations with implementing partners like you. This guidance just went live, I believe back in October so we’re still in the process
of socializing it. We first socialized it with USAID missions around the world and
we’re now reaching out to implementing partners to share the guidance, get
feedback on it and answer any questions that people have. So let’s dive
in. But before we dive into, kind of, the the changes, I think it’s really
important that we talk about why these changes have happened. There we go. And
really why USAID at this time has decided to release new education
reporting guidance is because of new, I say new but it was released in 2018, the
new U.S. Government Strategy on International Basic Education and the USAID Education Policy. I highly recommend if you have a chance to read these
documents, just to give you kind of a quick overview. The USG Strategy on
International Basic Education outlines the strategy by which
the various agencies of the US government are coordinating on
international basic education via the READ Act. So right now members of various agencies within the US government are meeting to discuss kind of how we do
jointly report on progress on international basic education to Congress. And the new USAID Education Policy outlines kind of the future and vision for USAID
education programming moving forward. For those of you familiar with the old USAID strategy from 2011 to 2015, with the old goal, one, two, and three. Those are out. We’re done with those. The Education Policy now prescribes and looks at a
much broader range of programming that USAID is going to move forward with over
the next few years. So when looking at kind of what the major changes are with
regards to what USAID is moving forward with, to respond to the USG Strategy and
the USAID Education Policy, there are five main things. And, we jumped a little bit ahead, so I’m going to jump back. So one is, again as I mentioned before,
the old goals are out. The old massive targets that USAID set,
you know 1 million children being able to read by a certain date. We’re now
changing our approach from a top-down approach to a bottom-up approach so
individual missions and operating units within USAID are going to be setting
their own targets against their education portfolios. And we’re currently
in the process of designing target setting guidance which will be released
in early 2020. This will be developed in consultation with many partners like
you, so look forward to that in the coming new year. Related, we’re also
producing guidance on internationally linked benchmarks, which I’ll go a little
bit more in-depth in a little bit later in the presentation. We also have new and
revised standard indicators across a variety of outcomes and outputs for the
education portfolio. Again I’ll jump into that as well a little bit later in the
presentation. And with those come revised and expanded disaggregates for those indicators and again, all this is designed to really
streamline reporting from the various activities that you all implement to the
missions themselves, and then up to USAID Washington for reporting out to
Congress, as well as kind of helping internal learning for the education
sector. So to dive in, to be a little bit more specific on our revised and
expanded indicators. So if you look at the USAID Education Policy, the vision
that is articulated there is that partner country education systems
will enable all children and youth to acquire the education skills needed to
be productive members of society. So in order to measure progress towards that vision, USAID has really expanded its indicators to encompass a
variety of different pieces. We now have indicators that span the full
spectrum of education levels, from pre primary to higher education, and measure
learning outcomes of a variety of skills, which include reading, math, social emotional learning, technical vocational skills. We’re
looking across a variety of populations, particularly marginalized and vulnerable
children and youth, because we’re very interested in a various countries
progress towards equity and inclusion. And finally, with that we’re also
expanding, as I mentioned before, expanding those indicators disaggregates
to measure across a variety of contexts including crisis and conflict affected
areas. So I’ve linked the list of standard and supplemental indicators here in the presentation and we’ll share this around after if you haven’t got a chance to take a look. But what we’ve come to are 25 standard indicators, of which 15 are
new. And if you look at the list and have had familiarity with education
indicators in the past, we’ve really expanded on the outcome indicators
because that’s really where we want to start focusing more on. Less about, kind
of, reach and more about what USAID programming is doing in each of the
countries that we work in. So I’ll dive into a couple of subgroups
of those indicators in a few moments. We also have what we’re calling
supplemental indicators. These are 16 supplemental indicators. These are
indicators that cover things that aren’t covered in our standard indicators, so
things like math outcomes, social emotional learning, school
readiness. Indicators that we’re in the process of testing out and expanding
upon to really complete the story of USAID programming. And so these are, for missions, these are indicators that we are encouraging missions to use instead
of custom indicators so that they can, again kind of complete that story of USAID programming for education. So to go a little bit more in depth
about what I’ve been talking about in terms of changes on standard and
indicators, just to take a look at a particular subgroup, one that I think
that’s particularly relevant is the reading outcome standard indicators. You
see the four main reading outcome standard indicators here. I won’t
read them, they’re up on the screen for you. They look probably somewhat
familiar from the previous guidance but they’ve changed in kind of, pretty profound ways. One thing to notice is we’re talking now about grade
level proficiency and what we’re really talking about there is setting
benchmarks as I mentioned previously. USAID right now is developing guidance on
benchmarking which is linked to the global proficiency framework that’s been
developed in conjunction with UNESCO. And what we’re, what the guidance, which again is coming in early 2020, is going to do is link that global proficiency
framework to a methodology for setting benchmarks that we call policy linking.
And I’m happy to go in-depth into the methodology if you’d like in the QA but
basically what this does is it allows countries to link their benchmarks to
international standards while using whatever assessment they like to use.
It’s really assessment agnostic. If countries would like to use the EGRA,
as is done in previous years, they can. If they have their other assessments,
they can use those assessments and link those assessments to this global
proficiency framework. The idea being is we really want to strengthen countries
systems to be able to properly assess reading at the end of
grade two, at the end of primary school. So that’s the big change in terms
of what we mean now by proficiency. We also now have an indicator specifically
for reading outcomes, learners with disabilities, meeting minimum grade
level proficiency ES.1-47. And this indicator is specifically designed for
programming that is specifically targeting learners with disabilities. Again we think it’s really important that we are
measuring progress around equity and inclusion and this is indicator’s one way that we’re going to be doing that moving forward, particularly for reading. We also have a suite of Youth Workforce
Development standard outcome indicators that have changed from the previous
guidance. Again, for those of you familiar with the guidance previously, they look a little familiar. One of the big changes
around both how we’re measuring changes in earnings following participation in
workforce development, as well as new employment, is there’s a new tool coming out. Actually, it has just
just come out, called the Workforce Outcomes Reporting Questionnaire, or as
we like to call it, the WORQ questionnaire. So this has just come out.
The guidance is coming out. A colleague of mine in the Office of Education,
Rebecca Pagel, is going to be reaching out to partners very soon to start
socializing this tool but this is going to be a tool that’s mandated for EG.6-11,
EG.6-12, to measure your new employment and earnings moving forward. We also have a new indicator focused specifically on improved soft skills. This again
highlights the growing importance of soft skills when looking at workforce
development and so that that is a new indicator that’s out right now. And then
we also have indicators looking at improved reading skills, ES.1-54, specifically for youth workforce development programming. This indicator
is designed to look at that in our supplemental indicators. We’re also
developing youth indicators for looking at technical/vocational skills, digital
skills and math skills as well. And so with all these new indicators
comes revised and expanded disaggregates. And again as I mentioned, it’s
really again, these disaggregates are focused on measuring progress towards
equity inclusion. So now what you’ll see in most of these standard indicators. You know previously, one of the mandated disaggregates was sex, and to a lesser
extent age. You’re now going to see that along with sex and age, measuring
persons with disabilities under those indicators, as well as crisis and
conflict affected individuals is going to become particularly standard for
indicators across the board. And again we think this is really important to get
a sense of how progress is going across different contexts and across
different populations moving forward. So we, as I mentioned before, we’ve been
you know. the guidance was made live in October. We’ve been socializing it with
the missions and one of the big questions coming back from the missions
is when is the appropriate time to transition to these new indicators.
Because missions are in various stages with different activities and
different stages in the USAID program cycle. What’s the proper time to be
shifting? So the Office of Education started to assign these new and revised
indicators for the most recent performance, portfolio performance
review for FY19, but what we’ve told missions is that ultimately they’re
going to have to determine how best transition to these new indicators based
on the respective portfolios. We recognize that for both missions and implementing partners, there are cost implications to transitioning to new
indicators. We understand that there are contract obligations that need to be met
so we’ve been really telling the missions that they should they should
talk with their implementing partners and see what’s most appropriate. However
what we’ve also told them is that for any new activities that are being
designed or for activities that are just starting up and developing their
monitoring and evaluation plans, that these new indicators should be incorporated
because we really are encouraging to move these indicators as fast as
possible so that we can start using them for our own learning as well as for
reporting to Congress. So really again we really just wanted to
give a very high-level overview and really what we want to hear from you
guys is your questions. So we’re gonna open it up to kind of an extended Q&A for you guys to pose your questions to me and to Rebecca. If you don’t feel
comfortable asking your questions now, or if you come up with a question later,
please feel free to reach out either through our official indicator helpdesk
at [email protected] You can also reach out to me directly. My email address is up there as well. I very much encourage you all
to visit the EducationLinks website. We have an indicator and
reporting guidance toolkit up there which Rebecca has done a great job
of putting together for you. That covers kind of a wide gamut of all the
changes covered I just described in my overview. So with that I’m gonna turn
it over to Rebecca and open it up to questions. Super, thank you so much, Chris. So we had a few questions that were
emailed in advance so I’m gonna start with those but I do want to invite
anyone to use the chat box right now to start entering any questions that you
might have and/or to quickly pull up that toolkit that Chris just put on the
screen. I know there was a question about if there were links that could be looked
at right now so go ahead if you need to and pull that up. We’d love to get your
questions in the chat. But Chris, one of the questions we received in advance is
about benchmarking. And this person says “I understand there are new requirements
for benchmarking. Are missions responsible for getting these in place
or do we need to build it into our plans? Does USAID have resources or tools that
can help with this?” So that’s a great question around benchmarking. So in terms of, you know, what missions, what missions should do around
benchmarking, the guidance in the PIRS for the various indicators that require benchmarking is that again we really are encouraging missions to look
at the global proficiency framework and and the policy linking methodology I
mentioned earlier to set those kind of international benchmarks. However missions and host countries aren’t bound to that. We recognize that
many host countries have set up their own benchmarks, are happy with those
benchmarks. So missions, in consultation with their host country government
and implementing partners can use those existing country benchmarks if
they prefer to report against the indicators. So really, you know, in terms
of who decides this, it’s really going to come down to consultations between a
host country government, a mission and their implementing partners. In terms of
resources, as I mentioned, we are in the process of developing again this
policy linking benchmark toolkit. It’s currently in draft phase and I believe
some of you may be providing some feedback on it. We’re going to be
releasing it in early FY 2020 so you’ll be seeing it early on in the new year. And
and that’s, that will be our kind of our main resource. And then as we
socialize that, you know, more resources will become available through the Office
of Education. Super, thank you, Chris. Our next question is about other
agencies and this person says, “Previously there was some alignment between reading
indicators for USAID and USDA’s McGovern-Dole indicators. Will that continue with these
new indicators?” So that’s a great question. That is linked to the work that we’re doing under the USG Strategy for Basic Education. As I mentioned, there’s
currently a working group of representatives from the various
agencies that work on international basic education including USDA. I’m
working with us to coordinate our various indicators for reporting to
Congress. So you know again with things like McGovern-Dole, we are looking at
that. We’re looking at how they how our indicators link to those indicators,
and then you know trying to create a kind of a coordinated response for
mission reporting. Super, thank you, Chris. We have a question
about the PIRS that has come in in the chat window. And they are saying, “Are all
the PIRS for the new indicators available?” And they’re asking
specifically about the one on teacher absenteeism, so maybe you can talk a
little bit about the ongoing development for some of the supplemental PIRS.
Definitely. So before I jump in the Supplemental indicators. For the standard
indicators, all the PIRS are available. And if for some reason they
aren’t, please let us know. For the supplemental indicators, some of the supplemental indicators are still in kind of development phase. We’re kind of
testing them, as it were and that’s why they’re, some of them are considered to
be supplemental. So on the EducationLinks, the link that I provided here, when you look
at the indicators, some of the supplemental indicators PIRS are not yet
available. And teacher absenteeism is one of them. We’re currently working with the DEEP team to finalize those PIRS and those PIRS will be coming in 2020. Super, thank you Chris and I did see the request in the chat window that we also
type the questions in as I’m saying them out loud so I will do my best to
multitask. So let me ask the question first and then I will put it in the chat
for you. Chris, our next question is kind of about some of these other policies
out in the world and they asked, “Can you talk about how these changes link to the journey to self-reliance and the SDGs?” Yes, so this is a question we get a
lot in terms of how our education reporting guidance links to those two
things. And they’re indirectly linked. So with the Journey to Self-Reliance, USAID has its own set of indicators specifically designed to measure a
country’s journey to self-reliance. For those of you who aren’t familiar with it, I highly recommend you reach out and
look at the USAID country road maps to the journey to self-reliance. Those
outline those very clearly, both the the seventeen primary metrics that USAID has come up with, as well as secondary metrics. So those are kind of
again at a country level separate from the specific reporting indicators that
we use for USAID programming which is what I’m describing here today. In
terms of the SDGs, USAID doesn’t report on the SDGs. What we do is we support
countries to support the SDGs. So for example, the proficiency framework I
described earlier is an attempt to help countries measure their progress towards
SDGs. So really again these indicators are designed, our indicators, our
education reporting indicators are designed to measure against USAID programming which helps our host country governments kind of think about how they report up against the journey of self-reliance and the SDG progress as well. Super, thank you, Chris. So Kelsey has entered a question in the chat and I will read it aloud for
you and then maybe she has a follow up we can let her come off mute. So she says, “I discussed the new indicators with my project CO and he said that we are
grandfathered in. Could you please confirm that or not for current USAID
projects? Is this mission specific?” So, Kelsey that’s a great question. So if
I understand your CO when he says that we’re grandfathered in and please
clarify if I get this wrong, what I interpret that it is, is that he’s got
basically the indicators that you have are linked with your various
contracts or agreements. And we recognize that that’s the case for many
missions for current activities that are ongoing. Again, what we’ve told
missions is that’s perfectly acceptable. Again if it’s, you know, if you’re looking
at indicators and an activity is far enough along that it doesn’t make sense
to modify a contract or an agreement or if there are
significant budget changes, then missions should feel comfortable continuing to
report against the old indicators. what are called archived indicators. Where
missions feel that they can, if an activity is early on in its development
or its progress, you know and missions and implementing partners feel
comfortable transitioning, we’ve encouraged them to do that. Again we’d
like to move to these new indicators as quickly as possible, but
that is very much mission specific. We have not given kind of a blanket response on current USAID projects. That being said, for anything
newer in design, we have told missions that they should start to put those
new indicators in. And Chris, that has sparked some related questions. Stephanie, if I get the nuance of this question wrong, please chime in and we’ll
maybe take you off mute. It seems like she is interested, Chris, in the
possibility of if they want to partly review their methods and maybe
incorporate some of these changes, would they talk to their USAID project officer?
What would that look like if they wanted to start making some changes sooner
rather than later? Stephanie that’s a great question. I would say again if on your side you know you’re looking at this new guidance, you want to
be better aligned, definitely take it up with your COR, AOR, you know have that conversation and see, you know between the two of you,
see what is appropriate in terms of in terms of transitioning. Again, you
know, I don’t want to give you, you know, one way or the other. I think again
it depends on the activity, the contract, your budget, a bunch of different things.
But I don’t see any reason why you know again if on your side, you’re
interested in aligning to the new, the new guidance, to raise it with your COR or AOR. Super. Chris, our next question is about
disaggregates, which I’m surprised that there have not been questions sooner. But
they’re looking specifically at EG.6-11, which is average percent change in
earnings, but I suspect this could be, you know, answered even more broadly. They’re saying there’s plenty of disaggregation levels. What’s the best
approach when you see so many disaggregates? So the first thing with disaggregates is again and it’s, this is a great question. But the first
thing that’s important to consider with disaggregates is, you know we’ve
listed everything. Everything that could be possibly relevant in what we want to
look at disaggregate wise. And we recognize that for some activities those
disaggregates aren’t relevant. For 6-11, you may not be working in crisis
in the conflict areas so you wouldn’t include that disaggregate. And we firmly
recognize that. From there, in terms of information, you know, information
being limited, again that’s a conversation to have with your AOR, or COR in terms of what you’re able to capture for that particular indicator,
for that particular activity. You know, we realize that you know some of this, some
of this information is disaggregates might be new for a particular
activity. And so again, it’s worth raising the conversation
with your COR/AOR to talk about kind of what makes sense and
how to how to move forward with that. Are there any other questions about disaggregates or anything else that we might want to hear from Chris about these new indicators these
new reporting requirements? Maybe even the policy? Chris, I assume you’re open to
questions about that too. Sure. Wow, does that mean there are no
questions? Well, if there aren’t, I’ll make one last plug again. I encourage everyone to please you know visit EducationLinks, look at the guidance, read through the PIRS. Again if questions come up, please feel free to email us and then you know in
terms of reaching out, to reach out through your missions, reach out through
your AORS and CORS, have the conversations. And again, we’re always
here to help. Just let us know. So with that, I think we can give everybody 20 minutes back, which is rare for us, so. Well, thank you very much, Chris
I think this is has been super helpful. Great, thanks so much, Chris. We would like to just gather any feedback you all have about today’s session. I’m going to post another little
poll here just asking you how how useful today’s session was and then please
offer us in the chat any suggestions you have about making something like this
more useful in the future or further needs you have related to the content. We would love to be able to meet those needs. So I’m going to go
ahead and post that poll. We appreciate your feedback and have a wonderful rest
of your day or evening. Go ahead, Sandra I see your note there in cha. Feel free to come off of mute or type into the chat.
Great, thank you. Probably easier if I just ask out loud. Hi Christoper, thank you so much and Rebecca and everybody. I was just wondering, on the workforce development
indicators, you know. I work at World Learning and we have some programs that work on workforce development. One of the main challenges that we have to measure
the increase in or more jobs or increase in salary is the, the time that
passes. You know, after the participants complete the program, sometimes they
don’t, they won’t get a job just the next month, right? And certainly their salary
increase sometimes won’t happen right right away. So how do you, how do we
calculate that entire measurement. And I apologize, because I haven’t read the PIRS, you know, like right now. I don’t have it on top of my head but I just wanted to see what’s your advice. So Sandra that’s a that’s a good question and I’ve worked previously on in my past lives before USAID, I’ve worked on USAID workforce development programs where I’ve had that same problem. Right. I don’t have a good answer for
you right off the top of my top of my head but what I would encourage you to
do is look out for again my colleague Rebecca Pagle here in the Office
Education is going to be doing a webinar on the WORQ questionnaire, the tool
that we’re using for or we’re going to be mandating for new employment. And that’s
a question definitely to pose to her. She’s our resident youth expert. I’ll
flag that for her now though. I’ll put that and kind of reach out to her about
that and if she has a good response I’ll make sure that she responds back to
you on that. Because it’s a really good question,
one that is I know challenging for many people who are doing this type of work. How to how to kind of adjust for that. Especially when there are percentages
you know because you have you know the denominator probably is what, people that
completed the training? Or completed the program? And then when do you measure? Right, like it’s not gonna be this quarter, next
quarter. It’s very interesting I really like it but it’s complex too, to
measure it but yeah okay, thank you so much. Yeah. You’re welcome.