ACID transactions

However boring the topic might be, I believe every developer everywhere should understand what an acid transaction is and know why they are important.

I am going to try and give a quick high'ish level description of each of the 4 ACID principals and try explain why they are important.

But first we need to know what a transaction is...

A transaction is a set of instructions that are carried out against something (for example a database) that takes it from one logical state to the next logical state

OK so with that in mind let's quickly discuss an example, when you withdraw money from your bank account at an atm the following needs to happen:
+ The money needs to subtracted from your account
+ The machine needs to give you the money
These 2 actions means your account goes from having one amount to the next.

ACID transaction rules tell us that for this to be a valid transaction the following should be true:

The transaction must be Atomic (Atomicity)

This means that a if any step in the transaction fails we need to fail all the steps.

In our example above we can't subtract money from the bank account and not give you the money. We also can't give you the money and not subtract it from you account(sadly).

The transaction needs to succeed or fail fully.

The transaction must have Consistently

Any system has rules, and consistency implies that for the transaction to be valid it needs to not violate these rules. It has to bring that data from one valid state to the next. If any rules or checks are not met the transaction should fail.

In our example, a rule might be that the account can't be overdrawn. If you request too much money the transaction needs to fail.

The transaction should be run in Isolation.

Isolation is a very well chosen word for this aspect of ACID. Basically you need to ensure that transactions execute as if they are executed sequentially, or put differently, not be influenced by each other during their processing.

Even if you queue all your requests and process them like that (which will violate the final rule of ACID btw) you can't be sure that they will arrive in a logical order, even when sent sequentially from the client. What's more is when multiple clients affect that same data then you might check the balance is correct for each before processing, but if the sum of the two transactions puts the balance in our example into overdrawn, then you have failed to keep your data in a valid state.

**so what then? **

Well there are various strategies like locking and conflict management that can help with this. In a simple database read write scenario your database server should take care most if it as long as you make sure you start and end your transactions, but it can get more involved when you have more than one player in the transaction, for example our little example, the database and the machine giving you the money are involved.

I just want to say that it is OK for a transaction to fail, as long as the last ACID principal is adhered to

The transaction must be Durable

Durability is often described from a success point of view: when a transaction is committed it should remain so no matter what.

I actually think it is more useful to think about the party requesting the transaction in order to understand this better. If you tell a user or service using your system or Api that a transaction is successful, then you better make sure it is. Ideally your system needs to be able to either confirm the success of the transaction before reporting a response. If it chooses to report the success early then it needs to have contingencies in place to deal with the failures. **You can't 20mins after a user or system did something suddenly pop up an error message. ** So either you log the error and deal with it some other way or your system requests the data to be resubmitted again as a seperate work flow.

As said before, it is OK for a transaction to fail but you have to tell the requesting party with confidence that it either succeeded or failed. Failing to do so can lead to duplicate requests if a success is reported as failure, or data loss if a failure is reported as success.

A note of caution, a valid request does to guarantee a successful transaction. Timeouts, conflicts or other unexpected system failures could result in the transaction failing. So be careful to report successful request submission as a successful transaction. If you do, make sure you have contingencies in place to deal with failures. They WILL happen!

When should I use ACID transactions.

Almost Always. Whether you are writing a website talking to a database or working an API dealing with millions of requests, not corrupting your data should be paramount.

The only situation I can think of when this is not required is in big data where all the transaction are inserts and reads. In this situation you want data flowing in and missing one or 2 events in the data stream is possibly not that big a deal. But make sure your big data implementation is OK with this loss.

Really hope that this helps someone out there. Please comment and let me know if there is anything I've missed or if anything is unclear.