AIs like GitHub Copilot can program – Is my job in danger?

Hi 🙂 I’m Ulrike, I’m studying media informatics in my master’s degree, and I’ve been a working student at Centigrade since my internship in the engineer department here last summer. Since artificial intelligence (AI) has become a hot topics, at least since the release of ChatGPT, some people are afraid that AIs like Midjourney (AI that generates images according to instructions) or GitHub Copilot (AI that assists in programming) might take their jobs. So we decided to take a look at it from the perspective of a junior developer. Should I be worried about my professional future, too?

Thus I have been experimenting with GitHub Copilot for the past few weeks and I’d like to share my experiences with you in this blog post.

Very briefly – what is GitHub Copilot actually?

Released in June 2022, GitHub Copilot is an extension for Integrated Development Environments (IDEs) that is trained as an AI with English-language text and public source code to help programmers with their work. GitHub Copilot was developed by GitHub and OpenAI (they also created ChatGPT) to combine the principles of pair programming with artificial intelligence to improve code quality. In addition, with GitHub Copilot at their side, programmers should be able to program faster and with less effort. This works through auto-completion: Based on the project structure and files, user input such as function names or by “translating” text-based comments into source code, GitHub Copilot generates code suggestions that the programmer can accept, reject or – if there are multiple suggestions – cycle through. To generate the programming recommendations, GitHub Copilot reads all the code in the project and sends it to the server, where the code is analysed using OpenAI Codex. Based on the results of this analysis, the code suggestions are then generated and displayed in the IDE.
This process works for over a dozen programming languages, with the recommendations for Python and JavaScript being the most accurate, because these two programming languages have the most publicly available source code for training the AI. In the following gif you can see quite well how the process can work:

A project to test GitHub Copilot

At Centigrade, we engineers mainly code web applications with Angular, so a lot of coding is done in JavaScript (TypeScript), HTML and CSS. As a frontend developer, I have the most experience with these “programming languages” and GitHub Copilot is supposed to work very well for JavaScript, so an Angular web app seemed ideal for testing. Unfortunately, I wasn’t able to use GitHub Copilot in the client project I was working on due to information protection and ownership issues, so I was wondering how best to try out programming with GitHub Copilot.

Creating a web app from scratch seemed like a bit too much hassle, and GitHub Copilot can only help me with programming and not with entering npm commands and project organisation. Therefore, I was looking for a simple web app that I could extend with some features using GitHub Copilot. I found what I was looking for in the official Angular tutorials: Angular Homes is a simple Angular app that displays a list of properties on the start page and provides more information about each property on a details page.

After getting the code working, adding sample images to the assets folder, and launching the application locally on my end, it looked like this – on the left, the home page with an overview of the properties, and on the right, the detailed view when you click “Learn more” on any

In the second step, I took a closer look at the app and searched for additional features that I could incorporate with GitHub Copilot. This led to the following tasks:

1. Display more than ten example items on the front page.
2. When you click anywhere on a property item, you should be taken to the detail view.
3. Each real estate object should have multiple pictures displayed in a slider on the details page.
4. When you click the “Apply now” button, the system should first check that all form fields have been filled in. If not, an error message should appear next to the corresponding field.
5. Once you have completed the application form and clicked on the “Apply now” button, you should be redirected to a new page where all the information about the application is listed.

To check later that GitHub Copilot “had thought” of everything, I wrote down key points for each task to indicate what the task involved. Since I had written down my key points in a markdown file and opened it in VS Code, I got suggestions from GitHub Copilot already there. Most of them made little sense; for example, here GitHub Copilot just copied the whole sentence and counted up after the plus signs, even though I hadn’t programmed anything:

Note for all screenshots: Text or code in white or other colours is typed by me, text or code in light grey are suggestions generated by GitHub Copilot.

But GitHub Copilot also helped me when I was unsure about the formatting commands. In the following examples, I couldn’t remember exactly how to create a divider or embed a link. So to test GitHub Copilot, I wrote a prompt which GitHub Copilot “answered” directly:

And even in the task analysis, GitHub Copilot sometimes understood directly what needed to be implemented later in the code:

It was really interesting to see what native language GitHub Copilot understands and how GitHub Copilot switches between German (notes) and English (task analysis) or mixes the two as in the screenshot.

Coding with GitHub Copilot

After the task analysis, I started coding. In the following section, I will briefly list the tasks and their subtasks and comment on how I fared when coding with GitHub Copilot.

Task 1: Create dummy data

Who hasn’t been there – to test something specific, whether the responsiveness of a website or the pagination of a list, you need more objects. It would actually be quick and easy to copy and paste, but unfortunately the objects have an ID that you have to match manually. Does GitHub Copilot do that automatically?

Yes, but unfortunately not in the way one would wish. In the Angular Homes app, the data for the real estate objects is in a JSON file that contains an array called “locations” with the objects. Each object has the following attributes:

In my ideal scenario, GitHub Copilot would have suggested a new object with a new ID and the same or similar content to the already existing objects as soon as I put a comma after the last object. Unfortunately, this was just wishful thinking, because in order to get a suggestion from GitHub Copilot at all, I had to open a new object and enter “id”. The value was then automatically generated, i.e. incremented, and the next attribute including value was directly suggested. However, I had to confirm the recommendations for each attribute individually, so this process involves hitting the tab key multiple times before a single new object is complete. To analyse whether GitHub Copilot can help in this way, we need to define the two procedures: On the one hand, you have to trigger GitHub Copilot and press Tab very often to generate new dummy data, and on the other hand, you have to copy, paste and change the IDs manually.

Comparing the two processes for ten objects, where only the IDs need to be incremented and the other attributes are unimportant, manual generation is significantly faster. However, it must be said that GitHub Copilot doesn’t behave consistently: When I later wanted to demonstrate exactly this case to a colleague, the programming assistant suddenly suggested entire objects with a new ID and all attribute values. A little later, I wanted to test the same case again, and it didn’t work anymore; GitHub Copilot only generated the ID as before and, after confirmation, each attribute individually.

Task 2: Forwarding to the detailed view

Most people are now used to getting further information about items on websites by clicking somewhere on the item.

Let’s take this example: The section from the menu of our website shown on the right. Personally, I expect to be taken to the “UX Management” content of the website, no matter whether I click on the font or on a particular point in the image.
So it made sense for me to move the routerLink from the “Learn more” button to the entire real estate object in the Angular Homes app. A simple task, really. But what does GitHub Copilot think?

At first, we seemed to have communication problems, because when I wrote my programming request in an HTML comment, I only got text as a suggestion, like you would write in a code comment about complicated places. Only when I wrote the command in my HTML code without a comment did I get a recommendation for implementation:

The code that GitHub Copilot suggests here is correct, but the AI gives no indication that the underlying line should be replaced by the programming recommendation. So when you accept the suggestion and then save, VS Code automatically adds a closing tag for the new section.

As you can see in the screenshot, we get an extra section in the section that should have been asjusted. This situation shows that you should not just blindly accept the GitHub Copilot’s suggestions, but also pay attention when programming with GitHub Copilot.

In addition, you must remember to remove the programming prompt for GitHub Copilot, otherwise the text will appear in the corresponding place in the user interface after compilation:

This is particularly dangerous in HTML, as neither GitHub Copilot nor the IDE will alert you to this, and especially short prompts can easily be overlooked.

After fixing these issues, I wanted to adjust the styling for the hover effect to test how GitHub Copilot performs in CSS files.

When the prompts are recognised, the recommendations work quite well, as you can see in the following screenshot:

Of course, GitHub Copilot suggests something as a value, and in the example even more than I wanted (colour), but I didn’t specify the background colour either. Besides, the “framework” is there now, and I can adjust the colour later.

I noticed that the suggestions are only generated if the corresponding class is explicitly added. As you can see in the example, it is not enough to simply add the prompt after the class to which the code should belong.

Instead, you need to consciously add the class affiliation “of the .data-container” to the request “change the background colour of the .data-container on hover”. Only then will you get a meaningful code suggestion, as you can see in the screenshot.

Task 3: Real estate images are to be displayed in a carousel slider

When you look at properties on a traditional marketing platform, ideally there are several images for each property that are displayed in a carousel slider. I wanted to include this in Angular Homes as well, even though each property would have the same sample images. GitHub Copilot’s text suggestions for the code generation prompt were very good; for example, it directly recommended that the slider should have arrows to navigate through the images. The HTML code generated from this was helpful in keeping track of what new HTML elements were needed, for example for the slider, but it was not functional.
Even after several new attempts, such as rewording the prompt or changing the dummy data and the interface from a string to an array of strings first, no working code was suggested – sometimes even worse than before. After some time of unsuccessful trial and error, I finally gave up and just for fun asked ChatGPT how I had to change the given code to get a carousel slider. Interestingly, ChatGPT had the same approach as GitHub Copilot, but that code didn’t work either, and even ChatGPT couldn’t find a solution either (even though the AI confidently claimed to have fixed the bug with the new answer each time…).

Task 4: Form validation

And there it is again, the occupational illness of identifying elements of a user interface that can lead to user frustration: When you click the “Apply now” button in the detail view, nothing happens at the user level – regardless of whether the input fields are filled in or not. So I decided to take advantage of this and first, add a form validation. This means that when you click on the “Apply now” button, it should check to see if any of the fields are empty. If this is the case, an error message should appear next to that field.

After implementation, the application form looked like this:

Working on this task, it was the first time I reached my limits in formulating the prompt. What I really wanted to say was the following: If none of the input fields has an empty or undefined value, the application should be submitted. No matter how and in which language (German and English) I formulated this statement, GitHub Copilot did not understand what I wanted from the AI. So I ended up programming this check myself and added the corresponding boolean variables. When setting the variables, however, GitHub Copilot again proved to be very useful: Once I had set the first variable with this.firstNameEmpty = false;, GitHub Copilot immediately suggested the same for the other form fields. Suggestions like this are very helpful because they save time and reduce typing.

Task 5: Check the application

Having been asked to check the correctness of the information entered in such important forms as applying for a property, I wanted to do the same here: When you click on the “Apply now” button, you should be taken to a new view that lists the data of the selected property and your own entries from the application form. For this I needed

• a new Angular component including HTML template and CSS file,
• a new route and
• a redirect from the detail view to the application review, which in inculudes the property ID and application data.

For whatever reason, I didn’t follow the order set out above, but wanted to create a new route in the routes.ts file before creating the new Angular component. Maybe it was something else, but as you can see in the screenshot, GitHub Copilot was only able to generate text for me at the time, instead of concrete code.

Later, when I wanted to test this case again for a non-existent component, it suddenly worked, and GitHub Copilot immmediately suggested a new route with path, component and title matching the given name. Apparently, the AI is constantly learning.

For all the other subtasks, GitHub Copilot saved me a lot of typing or copy-pasting with manual adjustments: Anything that followed a certain pattern during coding was automatically generated by GitHub Copilot, and all I had to do was accept the suggestions –whether I was programming in JavaScript or HTML.

The following examples show the results for passing the form input and reading it with the associated type, as required by TypeScript:

GitHub Copilot was also able to help me display the information in HTML:

I noticed that you cannot skip object attributes in HTML. So if you want to skip an attribute, you have to type the beginning or even the whole name of the next desired attribute. At this point, one could optimise the AI’s suggestions so that you can cycle through the attributes of an object, as you can when GitHub Copilot has multiple suggestions.

Personal conclusion

As we have seen, there are two different approaches to using GitHub Copilot:

On the one hand, you can “ask” GitHub Copilot questions by typing direct prompts in native language such as “change background colour on hover”, and you get an answer. Since GitHub Copilot was designed as a programming assistant and not as a question-answering AI like ChatGPT, you will often get incorrect results using this method. But once you get used to where and how to phrase the prompt, GitHub Copilot will usually suggest sensible code. Probably not always exactly what you are looking for, and also often in outdated notation (in JavaScript, for example, var vs. let), but even that is usually enough to give you an idea of how to solve a problem.
On the other hand, GitHub Copilot can be used very well for auto-completion. This saves the programmer tedious typing or copy-pasting with manual post-processing, for example when using objects and GitHub Copilot automatically fills in missing attributes – definitely a benefit for all programmers!

However, you still have to be careful when using GitHub Copilot and it is best to double check everything again manually. For some tasks, it may be quicker to program, or copy and paste the task yourself – then at least you know for sure that you have adapted all the variable names (or not).

Another problem with using GitHub Copilot is, of course, that your code is not analysed and processed locally by the AI, but sent to the server instead. This makes the use of GitHub Copilot impossible in companies like Centigrade, which are bound to customer projects and the associated non-disclosure agreements, etc.

But to get back to the original question: Should I be worried about my professional future because of programming AIs? Given the current state of AI development, definitely not! As long as AIs learn from faulty code and cannot perform tasks 100% correctly, we computer scientists will certainly not be replaced. I can well imagine that repetitive tasks such as unit testing or other tasks that follow an exact pattern will eventually no longer be programmed by us. In industry, the use of industrial robots to relieve humans of repetitive or dangerous tasks has been no different. But at the end of the day, at least for the time being, there is still a human being who has to check the work that has been done – whether in industrial production or in programming. How long will that last? Who knows.

But there is still a long way to go before humans are completely replaced by AIs – both in computer science and in the arts. Take this image generated by midjourney, for example. Apart from the fact that the prompt (roughly “An elephant drinking tea on a flying carpet”) was not particularly well taken, what mistakes do you see? As long as AIs produce something like this, artists are irreplaceable.

So, cheers to the professional path, that lies ahead of me! 🙂