I've managed to figure out what was the problem, so here goes:

Wireshark shown the following communication between client and server:

The WebRequest case

DELETE /booster HTTP/1.1Host: localhost:5984Connection: Keep-AliveHTTP/1.1 302 Moved TemporarilyServer: CouchDB/1.0.2 (Erlang OTP/R14B)Location: http://localhost:5984/_utils/session.html?return=%2Fbooster&reason=You%20are%20not%20a%20server%20admin.Date: Fri, 25 Nov 2011 09:54:29 GMTContent-Type: text/plain;charset=utf-8Content-Length: 64Cache-Control: must-revalidate{"error":"unauthorized","reason":"You are not a server admin."}DELETE /_utils/session.html?return=%2Fbooster&reason=You%20are%20not%20a%20server%20admin. HTTP/1.1Host: localhost:5984HTTP/1.1 405 Method Not AllowedServer: CouchDB/1.0.2 (Erlang OTP/R14B)Date: Fri, 25 Nov 2011 09:54:29 GMTContent-Type: text/plain;charset=utf-8Content-Length: 64Cache-Control: must-revalidateAllow: GET,HEAD{"error":"method_not_allowed","reason":"Only GET,HEAD allowed"}

The curl case

DELETE /booster HTTP/1.1Authorization: Basic YWRtaW46MTIzUser-Agent: curl/7.22.0 (i386-pc-win32) libcurl/7.22.0 OpenSSL/0.9.8r zlib/1.2.5Host: localhost:5984Accept: */*HTTP/1.1 404 Object Not FoundServer: CouchDB/1.0.2 (Erlang OTP/R14B)Date: Fri, 25 Nov 2011 09:54:14 GMTContent-Type: text/plain;charset=utf-8Content-Length: 41Cache-Control: must-revalidate{"error":"not_found","reason":"missing"}

It shows that WebRequest doesn't use Basic authorization, and curl does use one. Little bit of googling shown the way to use basic authorization on WebRequest and it looks like the following:

try{    var request = WebRequest.Create("http://localhost:5984/booster");    request.Headers.Clear();    request.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes("admin:123"));    request.Method = "DELETE";    var response = request.GetResponse();    Console.WriteLine(response.GetResponseString());}catch (WebException e){    Console.WriteLine(e.Response.GetResponseString());}