Buttermilk is more acidic than normal milk and usually works better with a raising agent to make the cake fluffier. If you use it on its own, the cake may become slightly denser due to the lack of reaction with a raising agent. ( Sciencey bit reacts with raising agent, creating bubbles and making extra light and fluffy, on its own it doesn’t)
You can add oil to the mixture instead of milk, but then you would have to make allowances for the additional liquid you’re adding and modify the recipe. (I don’t know the ratios because I’ve never done it, milk is my go-to).
Milk is by far the easiest solution, and what I used to give me really good results. I only added a few 1.5 tablespoons just to soften up the mixture, not a lot, but it gave me good results.
If you don’t want to add milk, then stick with the original recipe and just add the stock syrup to keep the cake moist. You can even add amaretto to the syrup to make an amaretto stock syrup if you prefer to do that.