Recently, I’ve been playing with the Wattpad API. I was interested in collecting stats about the most popular publications.

Even though the API is simple and well documented, all attempts to use it through a python script failed. Requests were rejected with a “403 Access forbidden” HTTP error. My first intuition was that the basic authentication was incorrectly specified, but despite the API samples, I couldn’t figure out what was wrong with it.

Using WireShark to sniff the API sample requests would have helped solving the issue, but since the API only supports https, sniffing wasn’t an option (I later realized that I could have simply disabled https, just to sniff the request, since I didn’t care about the response).

Eventually, the problem came from the User-Agent header. The python script used a default User-Agent, which was rejected by the Wattpad server. Impersonating a legitimate browser (ex : chrome) solved the issue, requests were immediately accepted.

Below is a Python sample.
Continue reading →