Location Icon Dublin, Ireland

AI Shifts Emphasis to Increased Testing, CI Pipelines, Simplicity

TL;DR: AI results in increased testing needing larger and more intelligent CI pipelines, a shift in focus towards test inputs and outcomes and to the creation of simpler and isolated components.

An AI-Assisted Change Made Me Think

I needed to modify a function. Instead of manually going through the code, I simply asked the AI to handle it. While AI is impressive, it makes many internal assumptions that often results in errors.

First, I asked the AI to write tests for the original function to ensure nothing would break. AI excels at writing tests, but not so with expected test outcomes. After getting several tests written, I verified the expected outcomes to ensure accuracy.

Next, instead of asking the AI to change the original function, I updated the test outcomes to my desired results and then asked the AI to update the code to let all the tests pass. Worked wonderfully.

Benefits and Limitations of AI-Generated Tests

Tests provide great confidence in using AI-generated code. I wouldn’t let AI modify the test inputs and expected results yet, but using AI for implementing tests and updating code saves a lot of time when properly tested.

This places results in having a large number of tests in the codebase as AI can generate tests easily, allowing developers to focus more on the expected outcomes rather than writing the tests.

Shifting Focus to CI Pipeline Development

However, increased testing creates a new challenge: large and cumbersome pipelines. Therefore, pipelines need to become smarter, run more selectively, efficient, simultanously, and more frequently than ever before.

This shifts a significant focus from software development to CI pipeline development and quality of it (e.g. CrowdStrike faulty test software).

Importance of Simpler Components for AI

As we leverage AI more extensively, it becomes crucial for AI to easily understand and update code without making numerous internal assumptions. AI performs better on simpler, well-isolated components, which align with established best practices like Separation of Concerns, the Single Responsibility Principle, etc.

High-quality code is paramount; if AI can easily comprehend and update your code more accurately, and it is thoroughly tested, it can significantly boost your team's productivity.

Bonus #1: Using AI with Mainstream Components

Using well-known tools, software libraries, and other software components can help AI understand and update code more accurately, while AI is likely to struggle with in-house tooling unless you train your own AI model for it. These mainstream components can be fully-featured software products to integrate and configure, simple libraries, or even programming languages and frameworks. The more widely adopted they are, the better AI will perform. Additionally, it's likely that the AI model you use might be better trained on the same company's other products.

Bonus 2: Streamlining DevOps Tools

I believe that DevOps, especially in setting up, configuring, and debugging mainstream tooling, is becoming increasingly easier with the help of AI. AI can quickly identify problematic areas in your operational setups through log analysis and assist in rapidly setting up various tools. While maintaining large clusters remains challenging, AI significantly simplifies the process as industry standards become more well-defined.