is "<script type='text/javascript'>" incorrect?

Where are the MIME types for <script> defined in the html5 W3C specification ?

Nowhere, it has a list (which includes some experimental and deprecated ones) but states that you can use any MIME type you like. MIME types are defined by IANA and text/javascript is officially marked as obsolete in favour of application/javascript

What is browser support like for "text/javascript" and "application/javascript" ?

Not good enough. There are still plenty of browsers around that don't recognise the latter. (This is, however, only a problem with the type attribute, you can set the HTTP Content-Type header correctly without worrying).

Which should be used ? Alternatively should we just not set type at all?

Since you only care about HTML 5, just omit the type attribute entirely. It is optional and the default language is JavaScript.

I think Brandon is (at least) partially wrong. The latest editor's draft of HTML5 (15 Aug 2011) says:

The type attribute gives the language of the script or format of the data. If the attribute is present, its value must be a valid MIME type. The charset parameter must not be specified. The default, which is used if the attribute is absent, is "text/javascript".

So if you do not include a type attribute, the default value is "text/javascript". If that is the default value, it must be a valid MIME type.

What are the other valid MIME types? The spec doesn't seem to give an example list, but it does specify the required format when it says:

A string is a valid MIME type if it matches the media-type rule defined in section 3.7 "Media Types" of RFC 2616

Which you can have the joy of reading here:

http://www.ietf.org/rfc/rfc2616.txt

Edit: Quentin is right: For HTML5, there's no need to include a type attribute, assuming you're using Javascript.