When AI exploded into prominence a few short years ago, it promised to revolutionise software development—or at the very least, take care of some of the more repetitive, time-consuming tasks involved.

But today, developers are still waiting for that assurance to become a reality. In fact, many professionals in the software development space are turning their backs on AI and its as-yet-unfulfilled potential. 

According to new data from StackOverflow, developers view AI less favourably than they did this time last year.  

Why the backslide? It could be that the more developers use AI tools in their software development process, the more flaws they see crop up. Disappointing results, inaccuracies, and security concerns could all be contributing to this apparent souring on AI tools. 

But despite this element of mistrust in current tools, the use of AI in software development isn’t going away. The latest figures suggest 81% of developers agree that AI can be hugely beneficial in increasing productivity. 

Development teams will need to implement them in some shape or form to stay competitive as AI continues to accelerate processes and move the needle on delivery times. With this in mind, how do we make sure that our AI tools are generating results that developers can trust?  

In this article, we’ll take a look at the state of developer trust in AI and outline some of the ways we size up AI outputs.  

Do developers trust AI? 

When Stack Overflow asked developers how much they trusted the accuracy of their AI tools’ output, the results were hardly glowing. Just 43% said they feel good about AI accuracy, while 31% are sceptical. Notably, these numbers are much the same as last year’s, showing that AI tools have made little progress in the accuracy of their results, or at least that is how it is perceived by end users.  

Interestingly, trust differs among developers with varying levels of experience. Developers just learning to code tend to put more stock in AI accuracy than their professional counterparts (49% vs. 42%), suggesting that developers with more years on the job are more dubious of AI tools.  

The study also found that developers are more willing to trust AI to perform particular tasks but less likely to trust it with others.  

A sizeable 82% of developers are using AI to write code, for instance. Searching for answers is another big use case, with 67% tagging in AI to do this. In other areas of the SLDC, however, usage drops significantly, with 31% of developers utilising AI to learn about a codebase, 27% to test code, and 13% using it to commit or review code.  

Despite the vast majority of developers surveyed stating that AI will become more integrated with their development processes within the next year, two-thirds stated that the biggest challenge they face when using AI tools is that they don’t trust the output or answers provided.  

What’s causing this dubious attitude towards AI tools? 

So why is faith in the accuracy of AI tools declining among developers? With so many AI development tools currently on the market, it’s difficult to pinpoint an overarching reason why these tools are letting developers down. But we can look at some of the most likely causes.  

Poor Understanding of Context 

While some AI tools can give accurate, complete answers to prompts they are general purpose, or specialise in a specific task, for example writing code. The problem here is that they, for the most part, don’t understand the specific context they are operating in when being used in an organisational setting. Organisational typically have standards when it comes to security, architectural patterns, or code quality. This leads to a lot of products offering suggestions which may be perfectly ‘good’, yet produce something unusable. This would obviously be very frustrating for any developer, and may lead to dissatisfaction in the product. 

Poor accuracy and relevance 

As many developers have discovered, AI tools often generate code or suggestions that are incorrect, inefficient, or just not applicable to the problem at hand. All this can waste developer time and introduce errors into the codebase. 

Poor prompt engineering can be one factor here. If a developer asks for code to implement a specific algorithm (like quicksort), without greater context, the AI tool might produce code for a different algorithm (such as mergesort) or even a completely unrelated function. 

Or, when asked to suggest improvements to an existing codebase based on performance metrics, a tool may provide recommendations that conflict with the overall code architecture or introduce new vulnerabilities. 

AI tools with limited knowledge bases can also be frustrating for developers. Tools that don’t have access to up-to-date or relevant knowledge can generate results that are obsolete, inefficient, or even violate coding standards. Some tools might even try to compensate and end up giving misleading, or incorrect suggestions, known as ‘Hallucinations’.. 

This limited knowledge is especially problematic when developing solutions for specific domains that require niche expertise. A tool with no access to data on medical image processing, for example, will only be able to offer generic solutions that are not suited for the unique job at hand. And this applies not only to industry-specific information but also to regulations and compliance standards, which an AI tool may unknowingly violate without the proper knowledge base.  

Explosion in ‘cash in’ tools 

Once Generative AI was pegged as the next big thing, the market was flooded with hastily developed AI tools. Now, as we sit at the very top of that AI hype cycle, countless untested AI tools are being sold to developers in an attempt to cash in on the trend.  

These tools often prioritise speed over quality, delivering subpar results that erode trust in the technology as a whole. The landscape has become extremely competitive and so organisations may be using tools and features that give sub-optimal results for fear of falling behind their competitors, or missing out on revenue streams. 

Capabilities of AI tools are often overstated, leading to a possible over expectation of their abilities by developers. Often, either due to a lack of capability on the part of the AI, or prompts that are not well engineered enough, these expectations are not met causing dissatisfaction on the part of the developer. 

Security concerns 

Developers handle a lot of sensitive information, and concerns about the security of AI tools and the potential for data breaches aren’t doing anything to endear them to software teams.  

There are big data privacy concerns around many AI tools too, with many developers worried that they might inadvertently share sensitive data with the AI tool that could be stored or processed without adequate security measures. Even the data used to train the AI model could contain sensitive information, which might be exposed if the model is compromised.  

They may also play a part in increasing the risk of supply chain attacks. If the AI tool or its underlying components are compromised, it could be exploited, with bad actors using this as a way to introduce malicious code into the software development process. 

Dependency risks 

Subpar AI tools can cause unnecessary dependencies to be introduced into the SDLC by recommending libraries or frameworks that are not strictly necessary for the project, increasing the project’s complexity and potential vulnerabilities. 

These dependencies might conflict with existing ones, leading to build errors or runtime issues, or prove to be outdated or insecure, exposing the project to vulnerabilities. 

Code generated by the tool might also introduce implicit dependencies that are not easy to spot, making it hard to manage and understand the project’s overall dependencies. 

Imagine a developer uses an AI tool to generate code for a web application, for instance. The tool suggests using a specific JavaScript library for handling form validation, which appears to be a good option. But while the library might be fit for the job, it also introduces several additional dependencies, increases the project’s size, and creates potential security vulnerabilities.  

The importance of accuracy and faithfulness in AI 

Building trust is critical to furthering the use of AI in software development and helping developers harness the benefits that it offers with complete confidence.  

The first step towards building that trust is to show that an AI tool is producing reliable and accurate results, while also looking out for any areas where improvements can be made. And the best way to do that is to measure some key metrics.  

Metrics to measure accuracy  

Precision: Measures the accuracy of positive predictions by comparing the number of instances predicted as positive with the number that are actually positive. This shows you how often an AI tool is correct when predicting a target class.  

For example, if an AI tool identifies 100 images as cats, and 90 of them really are cats, then precision is 90%. In this instance, the tool can identify cats, but has trouble determining what’s not a cat (a false positive). 

Recall: Measures the ability to find all positive instances and illustrates the completeness of positive predictions (in other words, how often the AI tool can find all the objects it’s targeting). 

For example, if there are 100 cats in a dataset, and the AI tool identifies 90 of them as cats, recall is 90%. In this instance, the AI can identify most cats, but not all cats (a false negative). 

F1-score: Calculates the mean of both precision and recall, providing a balanced measure of the AI tool’s accuracy and especially useful when both precision and recall are important. 

Accuracy: Measures the overall proportion of correct predictions out of the total number of cases, and gives you a general overview of performance by determining the percentage of correct classifications, both positive and negative.  

Metrics to measure faithfulness  

Hallucination rate: Measures the frequency with which an AI tool generates incorrect or nonsensical information.  

For example, if an AI tool generates 100 summaries, and 5 of them contain incorrect information, the hallucination rate is 5%. 

Consistency: Measure the degree to which the AI tool provides consistent outputs for the same input.  

For example, if an AI tool is asked the same question multiple times, it should provide similar answers every time, which will yield a high consistency rate.  

Adherence to constraints: Measures the extent to which the AI tool respects given constraints or requirements.  

For example, if an AI tool is asked to generate text within a specific word count, it should adhere to this limit. 

Other important metrics 

Some other metrics to help you track the overall performance of an AI tool include: 

Efficiency: Measures the computational resources required by the AI tool to produce results. 

Latency: Measures the time it takes for the AI tool to generate output. 

Explain ability: Measures the ability to understand the reasoning behind the AI tool’s decisions. 

Why measure AI tool metrics? 

By monitoring the performance of your AI tools using these metrics, you can pinpoint weaknesses in the tool and look into how you can improve it; if recall is low, for example, the tool might be missing relevant information.  

Analysing this data regularly can help identify trends and potential issues too. If the accuracy starts to decline, for instance, it might indicate a problem with the data or the model. 

This data can also help you compare the performance of different AI tools and models, so you can select the most appropriate one to level up your software development process.  

But perhaps most importantly, tracking and communicating these metrics to your team can have a big impact on developer belief and trust in AI tools. 

If you can showcase high accuracy metrics for an AI tool, particularly precision and recall, you can better demonstrate the tool’s ability to produce correct results and bolster developer confidence. 

Proving that a tool has a low hallucination rate and high consistency will show your team that an AI tool is reliable and trustworthy, and encourage them to incorporate it into their processes. 

Essentially, the more data-based reassurance you can give developers that a tool does what it’s expected to do (and that using it isn’t going to come back to bite them later on), the more likely they are to tap into the full range of benefits that AI can bring to the SDLC.  

And that’s only going to allow you to deliver better-quality products faster in the future.  

How NeuroLogik Ensures Trustworthy AI

NeuroLogik is an AI-powered application that streamlines software development, providing developers with instant answers to their technical questions, backed by always up-to-date documentation. To guarantee accuracy, reliability, and high-quality responses, NeuroLogik incorporates automated evaluation metrics designed to ensure consistent performance and trustworthiness. 

Find out more about NeuroLogik:

Learn more