Improving Diversity In Engineering Through Blindfolded Code Reviews At Rover
Below is an article originally written by Adam Stewart at PowerToFly Partner Rover, and published on May 9, 2018. Go to Rover's page on PowerToFly to see their open positions and learn more.
One of the many problems facing the tech world is a lack of diversity. At a previous position in my career, I was in a diversity training program with 32 people. Thirty of them were white males. Now that I'm at Rover, I'm proud to say that diversity is something we spend a lot of time and resources addressing and it's reflected in the diversity of our team. We feel that implicit biases and subjectivity in the hiring process should be actively addressed and, to the extent possible while still evaluating a candidate as a whole person, removed from the picture.
Borne from one of these conversations was the need to make our hiring process as anonymous as possible to remove bias, and we started by looking at the take-home coding challenge. As part of our normal engineering interview process, we give candidates a challenge to work on independently and return to us. Our goal with the take home coding challenge is to solely review a candidate's coding and design abilities, independent of the candidate's past work and years of experience. In theory, this is the place in our hiring funnel that can be completely scored based on technical skills.
To accomplish this, we realized we needed to revise the take-home project to be anonymous in order to mitigate any implicit biases a reviewer may have. With the full support of management, I added tickets for this to my sprint schedule. This had an impact on the velocity of my normal work, but it was the right thing to do. I worked with the hiring managers and other interested people across several teams to design and review the new process to get a system that would work for everyone.
Our old process
Before the anonymization project, when a candidate started the take-home challenge, we would create a new Github repository based on their Github username to maintain unique naming patterns. This project would be shared with the candidate for them to finish. Once the candidate was finished and had committed the last bit of code and documentation, they would notify the hiring manager.
The hiring manager would then get two engineers to review the code, and would add them to the Github repository. They would also add them to our hiring software as reviewers so they could input the results. This software sends out an email with deadlines and information. It also allowed the reviewer to see the resume, phone screen results, and some other information about the candidate in order to make a more informed decision. Once our internal review was complete, the results were submitted to the hiring manager who would make a decision on the candidate and if they should make it to the next step in the process.
Our new process
We decided there wasn't enough value in seeing the candidate's whole set of information (resume, name, years of experience, etc.) to justify letting any internal biases affect our decisions.
Therefore, we first removed the candidates Github username from the repository name. Now the projects only have a date and two initials as an identifier. This keeps the repositories unique without exposing the candidates choice of username.
Since some candidates leave identifiable information such as adding files based on their name, or signing their readme file, we needed to do something to prevent that. We now have instructions emailed to the candidate that inform them that they should not include any personal information in the project.
After the candidate is finished with the project, the hiring manager runs a short script against the candidates repository that replaces the candidate's username with the hiring manager's username. This preserves the log messages, which can be valuable, but obscures who actually did the commits.
The next difference in the process is that the hiring manager no longer adds the reviewers to the hiring software until after the review. The Github access is granted, and that is the only link the reviewer has into the candidate. The reviewer can now review the code as a standalone set of data without any bias.
The final change is that once the code review is complete, the reviewer submits the review in a new document that is sent to the hiring manager. The hiring manager can then copy and paste the review contents in the hiring software at this point, and complete this part of the interview.
This whole change in the process puts a little more workload on the hiring managers, but we felt it was important to implement. They now manually run an extra script to anonymize the Github repository, and they copy/paste the review into our hiring software. This only adds a couple of minutes to each candidate.
As a company, Rover is committed to hiring a diverse team. Diversity attracts more people and is a sign of a healthy and inclusive culture. It creates a more welcoming workplace. It can spark creativity and empathy, and allows the team to better help the people we serve. We are constantly trying to reduce biases in all aspects of our company culture, and an anonymized coding challenge is one small but necessary step towards that.
Diversity Reboot 2021: The One Hundred Day Kickoff<p><strong>When</strong>: February 1-5, 2021</p><p><strong>Where</strong>: Virtual</p><p><strong>Price to register:</strong> Free!</p><p><strong>Where to register: </strong><a href="https://summit.powertofly.com/" target="_blank">Here</a></p><p>We had to include our own Diversity Reboot on our list of the best diversity and inclusion events to attend in 2021 because we know firsthand how the quality of 100+ expert speakers, the enthusiasm of 10,000 participants, and the cutting-edge tech that enables meaningful virtual networking and job fairs combine to create a truly epic five-day experience. This year, the theme 100 Day Kickoff harnesses the energy of the new government's first 100 days in office to help jump-start personal and professional plans to build more diverse and inclusive workplaces. </p><p>Following the February summit, we'll have a monthly series of smaller virtual summits on topics spanning everything from returnships to LGBTQ+ advocacy, so be sure to stay tuned for updates!<br></p>
The Future of Diversity, Equity and Inclusion 2021<p><strong>When</strong>: February 3-4, 2021</p><p><strong>Where</strong>: Virtual</p><p><strong>Price to register:</strong> Free</p><p><strong>Where to register:</strong> <a href="https://www.hr.com/en/webcasts_events/virtual_events/upcoming_virtual_events/the-future-of-diversity-equity-and-inclusion-2021_kcxf8glq.html#detail" target="_blank">Here</a></p><p>This virtual conference put on by HR.com focuses on how social movements like #MeToo and Black Lives Matter have pushed DEI at work beyond legal compliance and into a major factor of any company or brand's culture, employee engagement, and performance. Topics include how to uncover and resolve pay gaps across your team and hire top-level diverse talent.</p>
Workplace Revolution: From Talk to Collective Action<p><strong>When</strong>: March 8-12, 2021</p><p><strong>Where</strong>: Virtual</p><p><strong>Price to register: </strong>$820</p><p><strong>Where to register:</strong> <a href="https://cvent.me/ZQ4BbE" target="_blank">Here</a></p><p>The Forum on Workplace Inclusion's 33rd annual conference includes 12 session tracks, from DEI Strategy to Social Responsibility, along with 59 workshops and daily networking sessions. This year's theme focuses on one question: "What will it take to start a workplace revolution that moves us from talk to action?"</p>
Diversity: How Employers Can Match Words With Deeds<p><strong>When</strong><strong>: </strong>May 19, 2021</p><p><strong>Where:</strong> Virtual</p><p><strong>Price to register</strong><strong>: </strong>Early bird registration is $49 and general admission is $149</p><p><strong>Where to register:</strong> <a href="https://hopin.com/events/may-virtual-conference-diversity-how-employers-can-match-words-with-deeds" target="_blank" rel="noopener noreferrer">Here</a></p><p>From Day One is hosting monthly conferences in 2021 focused on different ways for companies to foster strong relationships with their customers, communities, and employees. May's half-day virtual event is focused specifically on how companies can make diversity promises that don't fall flat and features workshops, panels, and a fireside chat.</p>
Hire with Diversity, Equity, and Inclusion<p><strong>When:</strong> August 18, 2021</p><p><strong>Where: </strong>Virtual</p><p><strong>Price to register: </strong>$195</p><p><strong>Where to register:</strong> <a href="https://www.hci.org/conferences/2021-virtual-conference-hire-diversity-equity-and-inclusion-august-18-2021" target="_blank">Here</a></p><p>This conference put on by the Human Capital Institute is one of 12 virtual conferences that HCI has planned for 2021. This one focuses on fair and inclusive talent acquisition, including how to attract diverse talent, implement inclusive hiring practices, and addressing bias in employee selection. Other conferences will focus on optimizing talent strategy, engaging employees, and developing your workforce.</p>
Virtual Grace Hopper Celebration 2021<p><strong>When:</strong> September 26-29, 2021</p><p><strong>Where:</strong> Virtual, broadcast from Chicago, Illinois</p><p><strong>Price to register:</strong> Was $799 for regular access to the virtual conference in 2020; 2021 pricing hasn't yet been announced</p><p><strong>Where to register:</strong> <a href="https://ghc.anitab.org/attend/registration/" target="_blank">Here</a>, though 2021 registration wasn't live at the time of writing</p><p>Grace Hopper might be the best-known conference for women in tech. Through keynote presentations, networking sessions, job fairs, and community-building activities, vGHC reached over 30,000 women for their 2020 conference and are expecting even more in 2021! While not a conference focused exclusively on diversity and inclusion, many speakers plan to focus their talks on creating environments for women to thrive in the male-dominated tech field.</p>
Inclusion 2021<p><strong>When:</strong> October 25-27, 2021</p><p><strong>Where:</strong> Virtual and in person in Austin, Texas as of now</p><p><strong>Price to register:</strong> Hasn't yet been announced</p><p><strong>Where to register: </strong><a href="https://conferences.shrm.org/inclusion" target="_blank">Here</a>, though 2021 registration wasn't live at the time of writing</p><p>The Society for Human Resource Management's biggest conference of the year saw 1,200 DEI leaders participate last year; SHRM hopes to see even more come to learn, be inspired, and to walk away with a playbook of implementable strategies to create truly inclusive workplace cultures.</p>
AfroTech 2021<p><strong></strong><strong>When:</strong> November 8-13, 2021</p><p><strong>Where:</strong> Virtual</p><p><strong>Price to register:</strong> Early bird pricing is $149 for individuals and $249 for corporate attendees; regular pricing hasn't yet been announced</p><p><strong>Where to register:</strong> <a href="https://experience.afrotech.com/" target="_blank">Here</a></p><p>AfroTech is a conference hosted by Blavity, a tech media platform for Black millennials. It focuses on emerging tech trends, connecting Black talent with top tech recruiters, and providing networking and educational opportunities, with an overall goal of building a strong Black tech community. Over 10,000 people participated in 2020. While the conference isn't focused specifically on DEI, its main audience of Black tech talent is an important one to understand and to engage at work and beyond, and several speakers plan to focus on issues of race and inclusion at work. </p>
A Conversation with Vouch's Lead Designer Carrie Phillips
Nina Unger, Talent Acquisition Specialist at SoftwareONE gave us a behind-the-scenes look at SoftwareONE's Application process, culture, and values.
Learn about the company and how you can make your application stand out!
To learn more about SoftwareONE and their open roles, click here.
We asked 30+ women how you can make 2021 your best year yet.
Resolutions are one thing. Goals are another.
How do you move from vaguely hopeful statements about what 2021 will mean for you personally and professionally to thoughtful plans that are likely to come to fruition?