Jude Assignments

With many thanks to Richard Hoshino and his team, and with the support from science fair regions across BC and the Yukon, we have developed a computer program that optimally assigns judges to science fair projects based on scientific topic, age group, and other criteria. This program is extremely flexible and can be modified based on the requirements of your specific fair.

Judge and project information is required in Excel format. Review this sample input file to see the format required for the automated judge assignments program. You can update this file with your own information and use it to assign judges with our program. A more detailed explanation of the input file and required format is below.

To access the judge assignments program, click here.

Any questions around the use of this program can be directed to info@sciencefairs.ca
For technical support, please contact Richard Hoshino at richard.hoshino@gmail.com

Input File Information

To use the judge assignments program, download the input file, make any changes you wish, and upload the file to optimally assign judges to projects. Then a whole bunch of calculations will happen in the background and within a few seconds, the website will produce three different Excel output files for you to download with the optimal solution (from the perspective of Judges, the Projects, and the Preference Score).

Divisions

• Each Division is given a unique ID (e.g. D01, D02, D03, etc.) chosen by the regional chair
• Each Division is given a unique Name (e.g. Elementary Earth/Physics) that is meaningful to the regional chair.
• For each Division, the regional chair can specify the minimum and maximum number of Judges that can evaluate the Projects in that Division.  This field can be left blank, if desired.
• For each Division, the regional chair decides the number of Judges that will evaluate each Project.  (Note that this number can be different for each Division.  For example, the chair can specify that each Elementary project is reviewed by 4 judges each, while each Senior project is reviewed by 5 judges each).
• For each Division, the regional chair can specify the minimum and maximum number of Projects that will be evaluated by each Judge.  (In this data set, taken from the Vancouver Island science fair, each Judge evaluates at least 4 and at most 5 Projects).

Projects

• Each Project has a unique ID, chosen by the regional chair.  To keep things simple my IDs were P001, P002, P003, … to indicate Project #1, Project #2, Project #3, and so on.  However, it’s perfectly fine to make the unique ID something else – e.g. using the student’s NAME.  Then the first column would have cells like this: Alice Smith, Bobby Jones, Caroline Anderson, etc.
• Each Project is assigned to exactly one Division.  This assignment is made by the regional chair (not my algorithm).
• Each Project has exactly one Language option – i.e., it is in English or French, but not both.
• Each Project has exactly one Age option – e.g. it is impossible for a Project to be both “Junior” and “Intermediate”.  The regional chair can specify what these Age categories are.  (In Vancouver Island, there are four Age categories).
• Each Project has exactly one Subject option – e.g. it is impossible for a Project to be both “Earth” and “Physics”.  The regional chair can specify what these Subject categories are.  (In Vancouver Island, there are six Subject categories).

Judges

• Each Judge has a unique ID, chosen by the regional chair.  To keep things simple my IDs were J001, J002, J003, … to indicate Judge #1, Judge #2, Judge #3, and so on.  However, it’s perfectly fine to make the unique ID something else – e.g. using the judge’s NAME.  Then the first column would have cells like this: Diana Smith, Edith Jones, Frank Anderson, etc.
• Each Judge has a single number (on a 1-N scale) measuring their Experience in judging Science Fairs in the past.  1 is a brand-new judge, N is someone with lots of regional/national experience, and the other numbers are in between.  The number N is chosen by the regional chair.  If the regional chair wishes to completely ignore this variable, they simply set N=1.
• Each Judge has a single number (on a 1-N scale) measuring their Education.  For example, 1 might be someone who is an undergraduate student, while N might be a fully tenured Ph.D. professor at a university.  Once again, the number N is chosen by the regional chair.
• Each Judge has a single number (on a 0-N scale) measuring their ability to evaluate projects in English and in French.  I strongly suspect that this variable will be irrelevant to most of the Regions in the SFFBC, since all of the projects will be done and marked in English.  In that case, simply set all the English and French cells to 1, which will make my program ignore this variable.
• Each Judge has a single number (on a 0-N scale) measuring their preference for evaluating Projects in a certain Age category.  Note that the Age categories (marked in YELLOW) must be completely identical to the Age categories provided in the Projects tab.
• Each Judge has a single number (on a 0-N scale) measuring their preference for evaluating Projects in a certain Subject category.  Note that the Subject categories (marked in GREEN) must be completely identical to the Subject categories provided in the Projects tab.  For example, there cannot be a Subject category called “Biotech” in one tab and a Subject category called “Biotechnology” in the other tab.

Required Assignments

• A Judge can be pre-assigned to a particular Division by the regional chair (e.g. Division Heads)
• A Judge can be pre-assigned to evaluate one or more Projects (this is particularly useful for subject-matter expertise for Senior Projects)

Forbidden Assignments

• The regional chair can ensure that a Judge is NOT in a particular Division
• The regional chair can ensure that a Judge is NOT evaluating one or more Projects (e.g. due to a conflict of interest)

Once this is done, my Python program optimally assigns Judges to Divisions, and then within each Division, optimally assigns Judges to Projects.  The key to the process is the calculation of the preference coefficient Pref(j, p), which specifies the preference of Judge j evaluating Project p.  To calculate this coefficient, I simply multiplied the rankings of that Judge applied to that Project.

For example, Project #1 is English, Elementary, Phys/Math
And Judge #1’s Experience score is 5, Education score is 2, English score is 5, Elementary score is 3, and Phys/Math score is 1.
Thus, the coefficient Pref(1,1) = 5 x 2 x 5 x 3 x 1 = 150.

Multiplication (not addition) is the right method to use, since a Judge can specify 0 for a category they don’t want or can’t have (e.g. French projects), and so by multiplying them the preference coefficient becomes zero.

Note that by specifying the preference coefficient in this way, more experienced judges will automatically get the Divisions they want, since they have higher preference scores.  This will ensure that the most experienced judges that desire the Senior Projects will indeed get to evaluate them, while inexperienced judges (e.g. undergrads) will be given lower priority to judge older age groups, even if that is their preference.

In the first two Output files, I provide the best assignment of Judges to Projects, and Projects to Judges, and do so in a simple timetable format so we see that each judge is evaluating (at most) one project in each timeslot, and each project is being evaluated by one judge in each timeslot.  You will see that ALL of the constraints are indeed satisfied.  I have also provided a third Output file that lists the preference scores of each (Judge, Project) assignment.  You can sort this file.

I attached a 10,000 point penalty to any (Judge, Project) assignment whose preference coefficient was zero.  You will see from the final column in that Output file why the preference coefficient was zero – e.g. because the Language variable was 0.  This will help the regional chairs, though my program is guaranteed to avoid zero-point assignments unless no other option is possible.