Alternative to GUID with Scalablity in mind and Friendly URL Alternative to GUID with Scalablity in mind and Friendly URL asp.net asp.net

Alternative to GUID with Scalablity in mind and Friendly URL


Collision Table

For YouTube like GUID's you can see this answer. They are basically keeping a database table of all random video ID's they are generating. When they request a new one, they check the table for any collisions. If they find a collision, they try to generate a new one.

Long Primary Keys

You could use a long (e.g. 275001120966638272) as a primary key, however if you have multiple servers generating unique identifiers you'll have to partition them somehow or introduce a global lock, so each server doesn't generate the same unique identifier.

Twitter Snowflake ID's

One solution to the partitioning problem with long ID's is to use snowflake ID's. This is what Twitter uses to generate it's ID's. All generated ID's are made up of the following parts:

  • Epoch timestamp in millisecond precision - 41 bits (gives us 69 years with a custom epoch)
  • Configured machine id - 10 bits (gives us up to 1024 machines)
  • Sequence number - 12 bits (A local counter per machine that rolls over every 4096)

One extra bit is reserved for future purposes. Since the ID's use timestamp as the first component, they are time sortable (which is very important for query performance).

Base64 Encoded GUID's

You can use ShortGuid which encodes a GUID as a base64 string. The downside is that the output is a little ugly (e.g. 00amyWGct0y_ze4lIsj2Mw) and it's case sensitive which may not be good for URL's if you are lower-casing them.

Base32 Encoded GUID's

There is also base32 encoding of GUID's, which you can see this answer for. These are slightly longer than ShortGuid above (e.g. lt7fz44kdqlu5pt7wnyzmu4ov4) but the advantage is that they can be all lower case.

Multiple Factors

One alternative I have been thinking about is to introduce multiple factors e.g. If Pintrest used a username and an ID for extra uniqueness:

https://pinterest.com/some-user/1

Here the ID 1 is unique to the user some-user and could be the number of posts they've made i.e. their next post would be 2. You could also use YouTube's approach with their video ID but specific to a user, this could lead to some ridiculously short URL's.


The first, simplest and practical scenario for unique keysis the increasing numbering sequence of the write order,This represent the record number inside one database providing unique numbering on a local scale : this is the -- often met -- application level requirement.

Next, the numerical approach based on a concatenation of time and counters is commonly used to ensure that concurrent transactions in same wagons will have unique ids before writing.

When the system gets highly threaded and distributed, like in highly concurrent situations, do some constraints need to be relaxed, before they become a penalty for scaling.

Universally unique identifier as primary key

Yes, it's a good practice.

  • A key reference system can provide independence from the underlying database system.
  • This provides one more level of integrity for the database when the evoked scenario occurs : backup, restore, scale, migrate and perhaps prove some authenticity.

This article Generating Globally Unique Identifiers for Use with MongoDBby Alexander Marquardt (a Senior Consulting Engineer at MongoDB) covers the question in detail and gives some insight about database and informatics.

UUID are 128 bits length. They introduce an amount of entropyhigh enough to ensure a practical uniqueness of labels.They can be represented by a 32 hex character strings.Enough to write several thousands of billions of billionsof decimal number.

Here are a few more questions that can occur when considering the overall principle and the analysis:

  1. should primary keys of databaseand Unique Resource Location be kept as two different entities ?
  2. does this numbering destruct the sequentiality in the system ?
  3. Does providing a machine host number (h),followed by a user number (u) and time (t) along a write index (i)guarantee the PK huti to stay unique ?

Now considering the DB system:

  • primary keys should be preserved as numerical (be it hexa)
  • the database system relies on it and this implies performance considerations.
  • their size should be fixed,
  • the system must answer rapidly to tell if it's potentially dealing with a PK or not.

Hashids

The hashing technique of Youtube is hashids.

It's a good choice :the hash are shorts and the length can be controlled,the alphabet can be customized,it is reversible (and as such interesting as short reference to the primary keys),it can use salt.it's design to hash positive numbers.

However it is a hash and as such the probability exists that a collision happen. They can be detected : unique constraint is violated before they are stored and in such case, should be run again.

Consider the comment to this answer to figure out how much entropy it's possible to get from a shorten sha1+b64 recipe.To anticipate on the colliding scenario,calls for the estimation of the future dimension of the database, that is, the potential number of records. Recommended reading : Z.Bloom, How Long Does An ID Need To Be ?

Milliseconds since epoch

Cited from the previous article, which provides most of the answer to the problem at hand with a nice synthetic style

It may not be necessary for you to encode every time since 1970however. If you are only interested in keeping recent records close toeach other, you only need enough values to ensure that you don’t havemore values with the same prefix than your database can cache at once


What you could do is convert a GUID into only numeric by converting all the letters into numbers in the guid. Here is a example of what that would look like. It's abit long but if that is not a problem this could be one way of going about generating the keys.

1004234499987310234371029731000544986101469898102

Here is the code i used to generate the string above. But i would probably recommend you using a long primary key insteed although it can be abit of a pain it's probably a safer way to do it then the function below.

    string generateKey()    {        Guid guid = Guid.NewGuid();        string newKey = "";        foreach(char c in guid.ToString().Replace("-", "").ToCharArray())        {            if(char.IsLetter(c))            {                newKey += (int)c;            }            else            {                newKey += c;            }        }        return newKey;    }

Edit:

I did some testing with only taking the 20 first numbers and out of 5000000 generated keys 4999978 was uniqe. But when using 25 first numbers it is 5000000 out of 5000000. I would recommend you to do some more testing if going with this method.