ChatGPT has been on everyone’s lips for last two months. I personally think, that it is a great piece of technology, that will definitely has it place in the software ecosystem. To find out how good it really is, I made a test.
I am a DevOps guy by heart and I have read a lot of twitter discussions and blog posts about how the ChatGPT is going to replace us all, because we are just mere humans. However I had my doubts about it and as I wanted to know my enemy better, I asked some series of questions from it.
The Interview
After short but serious conversation to really measure my upcoming enemy, I asked ChatGPT to make me an Azure DevOps release pipeline for Blazor application that is deployed into Azure App Service.
And I have to say that it did a really good job. It used variables, comments and named the jobs really. It even added some description about what the pipeline is doing. I never do that.
The Mistake
As I was trembling in my chair and sank deeper into the abyss of incompetence, I noticed one interesting thing in the YAML. The ChatGPT used AzureWebApp@1 task to deploy the app, but it didn’t provide the azureSubscription input which is mandatory. With this little boost of confidence I asked ChatGPT to refine the pipeline a little bit and to use App Service Slot swap (which is very popular app service feature) to deploy the app.
Ok it knows the slots too, damn it! Lets see what the final YAML looked like.
# Name of the pipeline
name: MyBlazorAppDeployment
# Trigger definition: Pipeline runs on every push to the main branch
trigger:
branches:
include:
- main
# Variables definition: Define variables that can be used throughout the pipeline
variables:
# Azure resources
azureSubscription: 'MyAzureSubscription'
resourceGroup: 'MyResourceGroup'
appServiceName: 'MyAppServiceName'
# Build and deploy settings
buildConfiguration: 'Release'
publishDir: '$(Build.ArtifactStagingDirectory)/wwwroot'
# Jobs definition: The set of steps to be executed in the pipeline
jobs:
# Job named "build"
- job: Build
pool:
vmImage: 'windows-latest'
steps:
# Checkout the code from the repository
- checkout: self
# Build and publish the app
- task: DotNetCoreCLI@2
displayName: 'dotnet publish'
inputs:
command: publish
publishWebProjects: false
projects: '**/*.csproj'
arguments: '-c $(buildConfiguration) -o $(publishDir)'
# Publish the built application to the pipeline
- task: PublishBuildArtifacts@1
inputs:
artifactName: 'drop'
pathToPublish: '$(publishDir)'
publishLocation: 'Container'
# Job named "deploy"
- job: Deploy
dependsOn: Build
pool:
vmImage: 'windows-latest'
steps:
# Download the built artifacts from the previous job ("Build")
- download: current
artifact: drop
# Authenticate to Azure and set subscription context
- task: AzureCLI@2
inputs:
azureSubscription: $(azureSubscription)
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
az account show
az account set --subscription $(azureSubscription)
# Create a new deployment slot with the name "staging"
- task: AzureWebAppDeployment@4
displayName: 'Create Deployment Slot'
inputs:
azureSubscription: $(azureSubscription)
appType: 'webAppLinux'
WebAppName: $(appServiceName)
ResourceGroupName: $(resourceGroup)
SlotName: 'staging'
# Deploy the app to the "staging" slot
- task: AzureWebApp@1
displayName: 'Azure Web App Deploy (Staging)'
inputs:
package: '$(System.ArtifactsDirectory)/drop'
appName: $(appServiceName)
slotName: 'staging'
resourceGroupName: $(resourceGroup)
# Swap the slots to make the "staging" slot the production slot
- task: AzureAppServiceManage@0
displayName: 'Swap Slots'
inputs:
ConnectedServiceName: $(azureSubscription)
Action: 'SwapSlots'
ResourceGroupName: $(resourceGroup)
WebAppName: $(appServiceName)
SourceSlot: 'staging'
Ok that’s quite impressive. Let’s see how well it works in real life and put it into Azure DevOps.
As it turns out, it doesn’t. The ChatGPT made up a completely new task called AzureWebAppDeployment. There is no task in Azure DevOps with that name. There is a task called AzureRmWebAppDeployment, so I think it meant to use that (we all do mistakes). Even after changing that, tweaking handful of parameters and fixing typos I still couldn’t get it running. It was just so far off, that I would have done it maybe faster from scratch, than spent all this time fixing missing parameters and wrong file paths.
Closing Speech
One big problem with ChatGPT is, that it looks really promising and credible. It can even made up its own tasks, that doesn’t exist and we believe it. However (and luckily for me), at this stage it cannot replace humans and I think that is not even it’s purpose. ChatGPT is a tool. Tool like IDE, Powershell or Google. It helps us to solve problems, but it doesn’t try to step into our shoes.
I will definitely keep eye on this project and see how it develops. It has great potential as a tool to support my DevOps career, but if it tries to take my place… it can try to.