Which is the best way to add a retry/rollback mechanism for sync/async tasks in C#?

Look at using Polly for retry scenarios which seems to align well with your Pseudo code. At the end of this answer is a sample from the documentation. You can do all sorts of retry scenarios, retry and waits etc. For example, you could retry a complete transaction a number of times, or alternatively retry a set of idempotent actions a number of times and then write compensation logic if/when the retry policy finally fails.

A memento patterns is more for undo-redo logic that you would find in a word processor (Ctrl-Z and Ctrl-Y).

Other helpful patterns to look at is a simple queue, a persistent queue or even a service bus to give you eventual consistency without having to have the user wait for everything to complete successfully.

// Retry three times, calling an action on each retry // with the current exception and retry countPolicy    .Handle<DivideByZeroException>()    .Retry(3, (exception, retryCount) =>    {        // do something     });

A sample based on your Pseudo-Code may look as follows:

static bool CreateAll(object1 obj1, object2 obj2){     // Policy to retry 3 times, waiting 5 seconds between retries.     var policy =         Policy              .Handle<SqlException>()              .WaitAndRetry(3, count =>              {                 return TimeSpan.FromSeconds(5);               });       policy.Execute(() => UpdateDatabase1(obj1));       policy.Execute(() => UpdateDatabase2(obj2));  }

You can opt for Command pattern where each command contains all the necessary information like connection string, service url, retry count etc.On top of this, you can consider rx, data flow blocks to do the plumbing.

High level view:

Update: Intention is to have Separation Of Concern. Retry logic is confined to one class which is a wrapper to existing command. You can do more analysis and come up with proper command, invoker and receiver objects and add rollback functionality.

public abstract class BaseCommand{    public abstract RxObservables Execute();}public class DBCommand : BaseCommand{    public override RxObservables Execute()    {        return new RxObservables();    }}public class WebServiceCommand : BaseCommand{    public override RxObservables Execute()    {        return new RxObservables();    }}public class ReTryCommand : BaseCommand // Decorator to existing db/web command{    private readonly BaseCommand _baseCommand;    public RetryCommand(BaseCommand baseCommand)    {         _baseCommand = baseCommand    }    public override RxObservables Execute()    {        try        {            //retry using Polly or Custom            return _baseCommand.Execute();        }        catch (Exception)        {            throw;        }    }}public class TaskDispatcher{    private readonly BaseCommand _baseCommand;    public TaskDispatcher(BaseCommand baseCommand)    {        _baseCommand = baseCommand;    }    public RxObservables ExecuteTask()    {        return _baseCommand.Execute();    }}public class Orchestrator{    public void Orchestrate()    {        var taskDispatcherForDb = new TaskDispatcher(new ReTryCommand(new DBCommand));        var taskDispatcherForWeb = new TaskDispatcher(new ReTryCommand(new WebCommand));        var dbResultStream = taskDispatcherForDb.ExecuteTask();        var WebResultStream = taskDispatcherForDb.ExecuteTask();    }}

For me this sounds like 'Distributed Transactions', since you have different resources (database, service communication, file i/o) and want to make a transaction that possible involves all of them.

In C# you could solve this with Microsoft Distributed Transaction Coordinator. For every resource you need a resource manager. For databases, like sql server and file i/o, it is already available, as far as i know. For others you can develop your own.

As an example, to execute these transactions you can use the TransactionScope class like this:

using (TransactionScope ts = new TransactionScope()){    //all db code here    // if an error occurs jump out of the using block and it will dispose and rollback    ts.Complete();}

(Example taken from here)

To develop your own resource manager, you have to implement IEnlistmentNotification and that can be a fairly complex task. Here is a short example.