Add params to given URL in Python

There are a couple of quirks with the urllib and urlparse modules. Here's a working example:

try:    import urlparse    from urllib import urlencodeexcept: # For Python 3    import urllib.parse as urlparse    from urllib.parse import urlencodeurl = "http://stackoverflow.com/search?q=question"params = {'lang':'en','tag':'python'}url_parts = list(urlparse.urlparse(url))query = dict(urlparse.parse_qsl(url_parts[4]))query.update(params)url_parts[4] = urlencode(query)print(urlparse.urlunparse(url_parts))

ParseResult, the result of urlparse(), is read-only and we need to convert it to a list before we can attempt to modify its data.

Why

I've been not satisfied with all the solutions on this page (come on, where is our favorite copy-paste thing?) so I wrote my own based on answers here. It tries to be complete and more Pythonic. I've added a handler for dict and bool values in arguments to be more consumer-side (JS) friendly, but they are yet optional, you can drop them.

How it works

Test 1: Adding new arguments, handling Arrays and Bool values:

url = 'http://stackoverflow.com/test'new_params = {'answers': False, 'data': ['some','values']}add_url_params(url, new_params) == \    'http://stackoverflow.com/test?data=some&data=values&answers=false'

Test 2: Rewriting existing args, handling DICT values:

url = 'http://stackoverflow.com/test/?question=false'new_params = {'question': {'__X__':'__Y__'}}add_url_params(url, new_params) == \    'http://stackoverflow.com/test/?question=%7B%22__X__%22%3A+%22__Y__%22%7D'

Talk is cheap. Show me the code.

Code itself. I've tried to describe it in details:

from json import dumpstry:    from urllib import urlencode, unquote    from urlparse import urlparse, parse_qsl, ParseResultexcept ImportError:    # Python 3 fallback    from urllib.parse import (        urlencode, unquote, urlparse, parse_qsl, ParseResult    )def add_url_params(url, params):    """ Add GET params to provided URL being aware of existing.    :param url: string of target URL    :param params: dict containing requested params to be added    :return: string with updated URL    >> url = 'http://stackoverflow.com/test?answers=true'    >> new_params = {'answers': False, 'data': ['some','values']}    >> add_url_params(url, new_params)    'http://stackoverflow.com/test?data=some&data=values&answers=false'    """    # Unquoting URL first so we don't loose existing args    url = unquote(url)    # Extracting url info    parsed_url = urlparse(url)    # Extracting URL arguments from parsed URL    get_args = parsed_url.query    # Converting URL arguments to dict    parsed_get_args = dict(parse_qsl(get_args))    # Merging URL arguments dict with new params    parsed_get_args.update(params)    # Bool and Dict values should be converted to json-friendly values    # you may throw this part away if you don't like it :)    parsed_get_args.update(        {k: dumps(v) for k, v in parsed_get_args.items()         if isinstance(v, (bool, dict))}    )    # Converting URL argument to proper query string    encoded_get_args = urlencode(parsed_get_args, doseq=True)    # Creating new parsed result object based on provided with new    # URL arguments. Same thing happens inside of urlparse.    new_url = ParseResult(        parsed_url.scheme, parsed_url.netloc, parsed_url.path,        parsed_url.params, encoded_get_args, parsed_url.fragment    ).geturl()    return new_url

Please be aware that there may be some issues, if you'll find one please let me know and we will make this thing better

You want to use URL encoding if the strings can have arbitrary data (for example, characters such as ampersands, slashes, etc. will need to be encoded).

Check out urllib.urlencode:

>>> import urllib>>> urllib.urlencode({'lang':'en','tag':'python'})'lang=en&tag=python'

In python3:

from urllib import parseparse.urlencode({'lang':'en','tag':'python'})