How Good Is ChatGPT at Programming and Will It Replace Software Developers?

Jan - Feb 2022

Has artificial intelligence become good enough to replace software developers?

ChatGPT’s natural language processing (NLP) and generation capabilities have been the talk of the town since creators OpenAI launched it in November 2022.

For the uninitiated, ChatGPT is a Large Language Model trained on massive libraries of online content, such as publicly available code and programming tutorials. The chatbot is capable of “searching” data sequences, analysing relationships between words, functions, and clauses, and output cohesive responses based on input prompts. 

Within three short months, ChatGPT has been used to build websites, games, and everything in between. And we haven’t even uncovered the full range of possibilities with it yet. The current version, GPT-3, runs on 175 billion parameters. But developers are already preparing to launch the 100 trillion parameter model, GPT-4

Does this mean that programming and software developing skills are now obsolete? Not quite. With a good understanding of what ChatGPT can and cannot do, software developers can integrate it into their workflow for a major productivity boost.

What programming tasks can ChatGPT do now?

Nearly everyone who uses ChatGPT for the first time will be impressed. All you need to do is key in a simple prompt to receive not just fully functional blocks of code, but code that is peppered with comments the way it would be if an actual software developer wrote it. 

You can see this clearly from an example on tech forum BleepingComputer, where ChatGPT was tasked to create a PHP program that scans for open ports on a hostname

On demand code looks amazing, but only an actual developer can tell if it’s right or wrong.

It can even be used to debug code that has already been written, a time consuming task that developers frequently encounter when refining their programs before launch. ChatGPT shortens the needle-in-a-haystack hunting process into mere seconds and will even explain why your code is wrong and how to fix it.

Replit CEO Amjad Masad using ChatGPT to debug a simple code sequence.

Whether it’s deploying software programmes or detecting security vulnerabilities, name a development function, and ChatGPT has probably done it. 

The problem? There’s no guarantee that ChatGPT’s output is actually functional or accurate. If you don’t believe us, take a look at what happens when it’s prompted to calculate a very simple math question:

ChatGPT answers definitely need to be verified by knowledge experts before being implemented.

ChatGPT can generate complex code while stumbling on simple math problems because it is only trained to know what reasonable answers look like. It doesn’t have the same understanding of code, math, or language like you and I do. 

In other words, actual software engineers who understand code structures are still needed to quality check whatever ChatGPT outputs.

What are the programming limitations of ChatGPT?

Since ChatGPT and its art generating counterparts DALL-E and Midjourney entered the scene, the human element has been mentioned a lot as the key differentiator that will help the average worker keep their jobs.

But what exactly is the human element? ChatGPT may be inaccurate now, but its algorithm will definitely be refined over time. That means there will come a time when ChatGPT will be able to generate code faster, and also more accurately than human software developers. 

That’s why it’s important to understand exactly what machines cannot do. That is, acting independently, empathetically, and creatively. 

ChatGPT won’t do anything unless prompted, which means that the code it produces can only be as good as the person who uses it. An amateur may use only ChatGPT to solve simple Leetcode problems, but an experienced software developer with deep understanding of computer science principles will be able to direct ChatGPT to create code that is maintainable, scalable, and meets the needs of users.

The ability to empathise is also highly underrated when it comes to building a business. Understanding things like user demand, user interface, business budgets, and technical constraints are essential for building software with features that can actually make it to the market, and be used by the masses. 

Finally, creativity is what has helped software developers come up with new products that solve existing problems. Sure, executing those ideas may be possible especially with the GPT-4 interaction, but someone will still have to come up with the idea, articulate it well, and assess ChatGPT’s output to ensure that the best product possible is made.

How will ChatGPT evolve in the future?

ChatGPT in its present state is still plagued by flaws. Its dataset is limited to information generated prior to 2021, and tends to overuse certain phrases and code blocks in its responses. For now, it can only assist with speeding up routine tasks and building skeletons for software developers to work on. 

For now.

Few people anticipated how ChatGPT would take the world by storm. There’s no telling how powerful the next iteration will look like once with increased data and improved algorithms factored in. ChatGPT could very well become even better at programming. Or it could be locked behind a subscription where its powers are only reserved for people who are willing to pay for it. 

One thing’s for sure, ChatGPT is here to stay. Rather than being afraid of it, time-starved software developers can take advantage of this (temporarily) free assistant to make their lives a little bit easier.

Keep your eyes on our TTAB page to be the first to hear about updates in tech, the job market, and the economy.

Get exclusive, first-hand insights from tech experts 👉 TTAB Career Conversation.

Receive up to $500 worth in reskilling subsidies by 👉 becoming a TTAB member.

More Insights

How Employers and Employees Can Guard Mental Health Care Amid Layoffs

Singaporeans in tech firms find themselves fearing for job security. Meta recently announced plans to...

17 Mar 2022

Read more

Proven Resume Writing Tips for Tech Interviews in the 2023 Job Market

Competition for coveted tech jobs has never been steeper...

Jan - Feb 2022

Read more

Will Tech Salaries Decline in 2023 Amid Tight Labour Markets & Record Inflation?

The job market has never been tighter...

Jan - Feb 2022

Read more