skip to 2016 update
Our users weren’t just active at hackathons – they were building great software projects during school, their spare time, or at work. A portfolio limited to hackathon projects produced a minimal reflection of our users.
Few sites attempted a similar solution, but those who did felt like half-hearted attempts instead of a thriving community. Did developers actually have a need to share their work? If so, could we solve this problem in an engaging way?
I needed to understand why developers share their work and how they were doing it currently to inform our unique portfolio direction.
I researched a variety of sites for developers to share what they’re working on, even if the primary objective wasn’t to build a portfolio (e.g., Stack Overflow). I also researched the designer market, where the portfolio model was thriving with sites like Dribbble and Behance.
Every Monday I sent a personal email to hackathon participants in the previous weekend’s hackathons, asking to chat about their experience using ChallengePost. Over a 6 month period, I conducted 45 interviews (15–30 minutes each).
“No time to put together a portfolio” ranked the #1 answer why developers weren’t sharing their work. For those who were, the most common methods were a personal website or GitHub.
Personal websites were appealing because of the level of customization possible, but they were rarely updated – most users I spoke with were embarrassed by its outdated content. The motivation was lacking in part because these sites existed in a vacuum, without a community to support projects with feedback.
GitHub was mentioned frequently despite its primary usage to collaborate on code. While GitHub profiles are often shared as part of a job interview process, we felt it wasn’t an accurate reflection of a developer, and found others who thought the same (e.g., Why GitHub is not your CV). A GitHub profile doesn’t allow customization (users can’t prioritize which projects are most important), and projects aren’t visible unless they’re open source.
We created a new “portfolio project” object, different from hackathon submission. It had a separate creation form, and users could add or update a project at any time. Hackathon submissions were converted into portfolio projects after the hackathon deadline so users could continue to edit them. (These 2 project types eventually became a pain point. See how we solved it.)
Version 1 of user portfolios put visual emphasis on the projects amid an otherwise basic design. I prioritized the development of key features before a fancy layout, and planned to experiment with more sophisticated design options in the future.
GitHub was a common part of user’s current workflow, so we created a “GitHub import” option for adding a project. It provided a great shortcut for starting a new project while highlighting the difference between ChallengePost and GitHub by showing the user what important fields they were missing after the import.
We launched a basic landing page to promote the new portfolio product to our users, focusing language on how we were complementary, but different, from GitHub. E.g., GitHub is for code, ChallengePost is for the story behind your code. This is where you “go beyond the readme.”
New users were directed to their portfolio after sign up. A welcome state helped them get started, which included a sample project and a profile completion meter to indicate what they were missing. This completion meter also appeared in the user dropdown menu for existing users who were missing key items (such as bio, skills, location, etc.) to entice them to visit and complete their portfolio.
Finally, I re-visited the portfolio page design, adjusting the layout to improve information hierarchy, include better mobile support, and introduce a custom header option to add some personality to the page.
Most of our metrics are monitored using Mixpanel, an event-based tracking service. We tracked multiple events around users adding projects to their portfolio to monitor drop off rates (what point in the submission process caused confusion?), which promotion efforts were most successful, if people were using the GitHub import feature, and more.
A quick survey asked for feedback after a user finished adding a project to their portfolio. I focused on the basics: why the user added a project, if they shared their portfolio, if they’d use it again, and how they’d improve the experience.
We ran this survey for about 1.5 years, generating 460 responses and filling our user interview pipeline – the last question asked if they’d be willing to talk with us further.
Hackathon participants often weren’t aware that ChallengePost was more than a hackathon platform (specifically that you could add non-hackathon projects to your portfolio), so this feedback helped us refine our marketing efforts to introduce the concept of portfolios earlier in the user lifecycle. As changes were made and time passed, it was gratifying to hear responses improve towards what we were aiming for.
GitHub was very popular with our users (including our own dev team) so instead of ignoring this fact, we embraced it. The GitHub import feature demonstrated we understood how our users worked, and helped us market our portfolios as the next natural step.
In addition to metrics and surveys, get on the phone (or gchat) with your users. An hour or so each week spent talking to users is invaluable research you can’t get any other way. It’s invigorating whether users are complaining about a feature that bugs them (I can’t wait to fix that!) or pitching a new idea. Best of all, hearing real people speak about your product informs the language you can use to sell it.
The profile completion meter was a successful solution to prompt users to visit their portfolio page, introduce new features, and encourage them to add missing info. We regularly monitored the metrics, and as the majority of users were educated about the new portfolio, action driven by this feature dwindled. At that point I made the decision to remove it, not because it was a failed experiment, but it had served its purpose and was no longer needed.
After the introduction of our jobs product, I revisited the objective of portfolios. Because they were created solely for our hackathon audience, they were missing key data an employer needs to review a potential candidate, such as work experience and education info. We needed to address these concerns for portfolios to support Devpost’s hiring-focused goals.
I started by auditing multiple legacy features, such as the social “following” component connecting users on our site. Were these elements still providing value to users, or distracting from higher priority features?
Less than 10% of users who were following someone interacted with the notification emails when “someone you follow did X.” Actions taken from those emails were negligible. It was eye-opening to discover this feature we promoted in prime real estate across our site wasn’t actually providing value. Diving into the metrics allowed me to form a compelling case to remove following from our platform entirely.
Streamlining the feature set helped me refresh the design, resulting in a portfolio focused on helping developers present the best representation of themselves to employers.
We've held off on implmenting this work due to other priorities, but I look forward to rolling these changes out.