Azure Data Factory: Publishing error
I ran into another generic error today while trying to publish changes to an Azure Data Factory pipeline. One of the things that really irks me with this tool is that the error messages are famously unhelpful. They don’t tell you what the problem is directly. Instead, they redirect you to unhelpful, generic documentation.
Here is today’s gem (I’ll show you how I found the actual problem):
Why? Why did it fail? Why did it not just spit it out plainly? Why does the document it references list a bunch of error codes and not tell me how to list the deployment operations for details?
I was frustrated, to say the least. Error codes do me no good if I don’t have an error code.
Figuring things out. On my own. Again. With Google.
I decided to search for “Please list deployment operations for details.” It yielded a much better document. This one was entitled: View deployment history with Azure Resource Manager. Note to Microsoft: this would be a much more useful link in the error.
I proceeded to figure out I had to go to the Azure Portal and do this:
- Navigate to the Subscriptions page.
- Open the appropriate subscription from the table.
- Scroll down to “Settings” in the menu and click on “Resource groups.”
- Open the appropriate resource group from the table.
- On the overview page, click on the Deployments “Failed, Succeeded” link. Mine looked like this:
- Figure out which deployment I was looking for, since they are all ridiculously named “publishing_<some number here>” and don’t bother to list what tool they came from. Thankfully mine was the most recent, so I clicked on that.
The operation details revealed the problem:
“Update of frequency or interval for tumbling window trigger is not allowed.” Considering I updated the interval for a tumbling window trigger, I knew exactly what the problem was. Now I can fix it.
A lesson in UI design (Ranting)
Aside from the obvious frustration I stated earlier, where the error was not given plainly, there is another looming issue here. Why did it let me change the interval in the trigger settings if that was an invalid action? All of this frustration could have been avoided if that was programmatically prevented in the interface.
I’m sure the product is complicated, and there is a lot to keep track of. Unfortunately these types of problems are all too common with Azure Data Factory. It is not a friendly tool to use.