Where the current focus on productivity is wrong

The focus on having more productive engineers is not the right way to go. I see companies struggle all the time defining what productivity even is, and then they still tend to focus on lines of code accepted as a sense of productivity. Us engineers are busy all day with tasks like requirements engineering, architectural work, documentation, and discussions on a way forward. We think about maintainability, readability, and testability of the code we write. And then in most companies we also have all the extra things next to producing code like daily meetings, stand ups, check-ins, and so on. We hope that we get enough time to actually produce some value for our end users! I even met an engineer that starts their day later on purpose, so that they can continue on working when everyone has already left for the day: that is the time they can actually focus on writing code!

We have learned several times over the last decades that just looking at things like lines of code produced is not even close to a good metric to measure productivity. And even worse: the Copilot metrics API only shows the lines of code accepted for suggestions that where accepted as a whole! So if Copilot suggests 10 lines of code and you only accept 5 (or a couple of words), the API will count this as 0 lines of code accepted. So partial acceptance is not even counted in the metrics.

Still, companies want to look at lines of code suggested and accepted, and I even have seen companies that want to focus on driving that acceptance rate up! I think that is the wrong way to go, as it will lead to engineers blindly accepting suggestions without thinking about the code that is being suggested. Looking at a suggestion and thinking if this is the thing you want to do or not, or maybe change the direction you’re going in, is a good thing! GitHub is already showing information in their API to indicate the amount of suggestions that where shown, and the amount of suggestions that where accepted (including partial acceptances). This is already a step forward, and helps you to focus on engagement instead of productivity.

What to focus on instead

I recommend that people instead focus on other things instead of lines of code to see if the use of GitHub Copilot has an impact. This includes looking at the entire Software Development Life Cycle (SDLC):

• first of all you have to look at engagement and type of engagement of the engineers. Are they actually using the tool, and what kind of features are they using? I keep on running into engineers that only use the inline suggestions, and they’re not using the Chat feature at all! I find myself living in Agent Mode all day long, as I tend to know how to prompt for the right things to get the right changes, instead of building everything myself. I am thinking in business value, instead of code.
• also look at when they use the tool. We keep on thinking that engineers work 8 hours a day, 5 days a week (depending on their schedule), but in reality they are only productive for a couple of hours a day. Sometimes they even struggle to have 2 hours a day of uninterrupted time to work on their tasks! Seriously, ask you engineers how much time they have to work on their tasks. I see team mates with overloaded agendas, having multiple meetings scheduled in parallel all day long. No wonder they cannot focus on their tasks! This fact alone amazes me every time, as businesses are still wanting to define and show the Return on Investment (ROI) of spending time on a tool like GitHub Copilot. The normal Business license is 19$ a month, and the Enterprise license is 49$ a month. If you calculate that you need to give the engineer an improvement and let them save time for around 1 hour a month to break even, you can see that the ROI is easily achieved. Of course this does not include the initial training time (there is quire a learning curve in my opinion!), but still overall, the ROI is easily achieved.
• next is taking a look at the downstream changes in the SDLC. Are there more Pull Request created then before? Are the PR’s smaller or larger in size then before? Are there more or less build failures (indicating that users are testing in the pipeline, instead of testing locally)? Do you find more or less bugs in production? Are there more or less incidents in production? All these things indicate what the effects are of using GitHub Copilot in your organization.
• we also keep forgetting that having these tools available for your engineers is almost expected these days. It is becoming a normal thing, just like having a decent environment available, together with other tools like a good IDE, a good version control system, and a good CI/CD pipeline. If you do not have these tools available, you will have a hard time attracting new engineers to your organization, and even worse, the existing talent might leave your company because of it. And you can better believe that your competitors are already offering these tools to their engineers! So can you even afford to be falling behind in this area?
• next up is that we are supposed to work faster on the boring parts of our jobs, where we can implement default things like boilerplate code easier and faster. The goal here is not to be more productive in my opinion, but to get your valuable time back to work on the more interesting parts of your job! Use the time you save to work on the things that are more interesting to you, or the things you usually do not have time for. I recommend creating initiatives especially during the training phase of using Copilot to focus on those tasks that you never seem to have time for. Let Copilot help in the next two sprints with having better test coverage. Ask Copilot to locate places that can be refactored into more maintainable code, or more readable code. You will be amazed on the things it might find for you. I dare you to use Agent Mode for example and ask it for find all your TODO’s in the codebase and address them one by one (or create Issues from them by using the GitHub MCP server). Triage the low hanging fruit with Copilot and ask it for a fix. Don’t have a linter in the project yet? Add it and ask Copilot to fix all the linting issues it finds. Keep forgetting to update your dependencies? Ask Copilot for a Dependabot config for all the ecosystems you have in your repo. Always wanted to run your linting/unit tests in a CI (Continuous Integration) pipeline and never got to it? Guess what… Copilot can help here! The possibilities are endless, and you can use Copilot to help you with the boring parts of your job.

What’s next?

I jotted down how I think we need to change the way of thinking around productivity in this blogpost. Then we are liberated to start thinking about the way GitHub Copilot changes the way we produce our applications and bring actual value to our end users. And even better, how we can really level up and also enable our non-engineering team members to work more efficiently as well. I have written more about this in this blogpost GitHub Copilot - Change the Narrative.

Lets work in the right direction and embrace the future state of working by leveraging Generative AI more and more, and leave the past behind us. I think we will become more of an orchestrator of AI agents, and the agents can only go faster if we understand how to build a solid foundation to trust on (see that blogpost).