I'm not sure LLMs will be good at doing taxes any time soon primarily because they're very bad at even simple math unless it's explained in words like in proofs. A good illustration of this is asking an LLM something like this:
"If I put a container on a food scale, tare it, add 100g of peanuts, tare it, weigh 100g of almonds and then remove the container from the scale, what does the scale read?"
LLMs completely puke all over themselves on this one. They do not account for not taring after the almonds and they do not see the weight of the container needs to be subtracted. The correct answer is minus (100g + container weight). Although this particular question also exploits another LLM weakness which is physical world concepts. LLM are just guessing. For an LLM to do taxes well, it would need to be trained on many scenarios and it would need a math service that it can use as a support service.
You just need to give an instruction something along the lines of "any initial calculation or derived value must be computed through the use of a $LANG program."
Pretty sure the LLMs could implement tax calculations better than the average accountant, were they given the rules.
