google still have the crown jewels in the AI arms race

Although Microsoft has Github Copilot, and has embedded the same AI technologies into Office 365, it appears Deepmind is providing the major scientific breakthroughs.

The scientists at Deep Mind recently applied the same technology that worked out how to play Go and Chess to finding a better hashing algorithm, amongst other things. Deep Mind managed to create its own Algorithm which was 30% faster than any technique invented by a human. More about this in this article from Nature.

Algorithms are a big subject and many fine minds over the years have applied themselves to the various Computer Algorithms, one of which is hashing. I once bought a book on "Introduction to Computer Algorithms" and was somewhat disheartened to learn that this "introduction" was 1,300 pages long. 

Deep Mind is at the forefront of a branch of AI called Unsupervised Learning. It trains itself by trial and error to achieve certain goals.

GPT4 and ChatGPT are fundamentally different technologies to Deepmind. GPT4 and ChatGPT are examples of Large Language Models or LLMs. LLMs need training data to provide results. In the case of Copilot it is trained on hundreds of thousands of Github repositories.

If the data, ie the quality of the coding techniques being used to train Copilot are good, then the computer code suggested by Copilot will be good. However if the bulk of the code being used to train Copilot is just low quality, non-performant code, then the suggestions given by Copilot will be of the same standard. The writer of this article in Nature magazine in using Github Copilot is very similar to my own experience. The suggestions given by Copilot are sometimes useful but always have to be treated with some caution.

The next challenge for Copilot would appear to be how to continue to use LLMs to understand the grammar and syntax of computer languages, but then to somehow combine Unsupervised Learning techniques to actually use that code to understand and solve problems.  

 

 

---