using AI to build software and how to avoid the race to the average

Despite the complexities of AI, there is a simple conclusion to be reached when using AI to generate code. The code suggested by the AI will be the code most commonly used, but not necessarily the best code to solve the problem.

The overwhelming majority of code in public repositories was never meant for production use, and such code provides much of the training data for Copilot. Of course that's not all there is to training the Codex model which powers Copilot, however it is a key reason why the most likely suggestion from Copilot is code that might do the job but not necessarily be the most performant.

That's great if you are a beginner - you get an easy leg up into being a functional developer. For an experienced programmer however, the code suggested by the AI might not be as good as the code they could write themselves with just a bit more research.

Let's take the example of creating an immutable type in C#. That is, an object that can't be changed after it gets created.

The task I posed to Copilot was :

"Create an immutable type called Person with two properties - FirstName and LastName".

Now there are tens of thousands of repositories in Github that contain code and comments similar to the following:

//creating immutable type called Person using readonly properties

public class Person

{

    public string FirstName { get; }

    public string LastName { get; }

}

The comments above the code provide the pointer that Copilot uses to return the answer to the question. Sure enough, the above code was exactly what Copilot returned when the above question was posed. I tried a dozen different ways of coaxing it towards a better answer. The response from Copilot however, was always the same.

Until 2020, the above code was the best way to create immutable types in C#. However, in 2020 the record type was introduced. For an object like this with no other functionality, and only a few properties, using a record is better than using a class.

So I gave up coaxing and instead asked the following question using Copilot Chat - "wouldn't using a record be better in this case?"

The answer:

Out of interest, the next day I posed the same question with respect to creating another object. At this time, unlike the previous day, I had a few other files open in Visual Studio that contained definitions for record types. This time, the suggested answer was indeed to use a record. In this case, Copilot was also looking at the other open (record) files to add additional information to the prompt that I typed in.

This demonstrates the importance of providing as much detail in the prompt as possible, in order for the AI to give the most accurate result. An experienced programmer, knowing the details of the context, will be able to provide the best prompt in order to get the best result. Inexperienced programmers however, are more likely to just receive adequate code that simply does the job.

---