GitHub introduced Copilot Chat, a programming-focused chatbot similar to ChatGPT, only for enterprises subscribed to Copilot for Business. The beta version of Copilot Chat was just made available to individual Copilot users who are subscribed to the $10 per month plan. GitHub is currently making chat available to all users.
Starting today, Copilot Chat can be accessed in the sidebar of Microsoft’s IDEs, including Visual Studio Code and Visual Studio. This feature is included in the subscription tiers of GitHub Copilot but is also accessible for free to verified professors, students, and maintainers of certain open source projects.
“Being the hub for developers worldwide, we have successfully introduced the most extensively utilized artificial intelligence developer tool in history,” said Shuyin Zhao, Vice President of Product Management at GitHub, during an email interview with Eltrys. “And the completion of coding was merely the initial stage.”
There have been few changes to Copilot Chat since the beta version.
The leading generative AI model from OpenAI, GPT-4, is now powering the chatbot. It has been specially fine-tuned for development settings. Developers have the ability to initiate Copilot Chat using natural language in order to get immediate help. This includes requesting explanations of ideas, identifying vulnerabilities, or generating unit tests.
The Copilot Chat’s underlying model, GPT-4, like other generative AI models, underwent training using publicly accessible data, which includes copyrighted or restricted licensed content. Vendors such as GitHub contend that the fair use theory provides them with protection against copyright lawsuits. However, this has not deterred programmers from initiating class action lawsuits against GitHub, Microsoft (the parent firm of GitHub), and OpenAI, claiming that there have been infringements on open source licenses and intellectual property.
I inquired with Zhao about the possibility for codebase owners to exercise their choice to abstain from participating in the training process at this time if they choose to do so. She said that there is no novel method in place for this with the wider release of Copilot Chat. Instead, she recommended that codebase owners privatize their repositories to avoid their inclusion in future training datasets.
I anticipate that codebase owners will not react well to that proposal. There are several justifications for maintaining copyrighted code in the public domain, with one of the least significant being the opportunity for collective bug detection. However, GitHub is clearly unwilling to compromise on allowing users to opt out of training data, at least for now.
Generative AI models, such as GPT-4, also have a proclivity for hallucinations when they confidently fabricate information, which poses a significant challenge in the domain of coding. Based on recent research conducted by Stanford, developers who use AI helpers for coding tend to generate code that is less secure in comparison to those who do not employ AI assistants. This is mostly due to the fact that the AI assistants introduce faulty or outdated code snippets.
Zhao stated that GPT-4 demonstrates superior performance in addressing hallucinations compared to the previous model that previously powered Copilot. Zhao also highlighted the exploit-mitigating capabilities of GPT-4, including filters for insecure code patterns. These filters alert Copilot Chat users about vulnerabilities such as hardcoded credentials, SQL injections, and path injections. However, she emphasized the need for a thorough human evaluation of any code provided by AI.
“According to Zhao, GitHub Copilot utilizes OpenAI’s models, which have been determined to be the most superior models for our current range of services,” Zhao said. “We are currently well-positioned to further enable developers with the necessary AI tools to construct superior and more secure software on a large scale, all while enjoying the process.”
During an analyst briefing in October, Satya Nadella, the CEO of Microsoft, said that Copilot had amassed a user base of 1 million paying customers and around 37,000 business clients. However, it is essential for GitHub to enhance the appeal of Copilot in order to avoid losing market share to rivals and, therefore, losing revenue.
The Wall Street Journal reported that Copilot loses $20 per user per month on average, with some clients costing GitHub up to $80 per month. The exorbitant cost associated with operating the underlying AI models is said to be the cause of this issue. Kite, a company that specializes in GenAI code, also encountered this issue, which ultimately caused its premature closure in December of last year.
While GitHub has challenges in making Copilot financially successful, Amazon is actively enhancing CodeWhisperer, which is considered to be Copilot’s most well-funded competitor.
In April, Amazon implemented a policy where developers may use CodeWhisperer at no cost and without any limitations on its usage. In addition, during the same month, CodeWhisperer Professional Tier was introduced, including single sign-on functionality with AWS Identity and Access Management integration as well as expanded capabilities for scanning and detecting security flaws. The CodeWhisperer business plan was introduced in September. In early November, Amazon made improvements to CodeWhisperer to provide more advanced recommendations for app development on MongoDB, the open-source database management software.
