Quick Background
I had made an earlier post regarding how specifically I have been having trouble porting over my python hypemachine script into Objective-C. Being new to Objective-C and Cocoa development specifically I wasn't sure whether I was doing something stupidly wrong or that the hypemachine guys were doing something to thwart my pet project. Turns out it was the latter, so with a new resolve to figure out the web I moved on!
Hello Wireshark
As mentioned many times already, I rarely do web development or anything involving networks. Therefore figuring out what was happening was a great way to get some new exposure to some tools and re-familiarize myself with how things are done. (It's been a while since my networks course). A good place to start I knew was to inspect the TCP/IP packets using Wireshark.
Wireshark is a network protocol analyzer. It lets you capture and interactively browse the traffic running on a computer network.
With Wireshark all setup, I began to inspect the HTTP packets. Knowing that obviously requests through the browser work and those from my python script work I stored the HTTPHeaders of all 3 to scan for differences.
~Python~
Severity level : chat
Group: Sequence
Request Method: GET
Request Version: HTTP/1.1
Accept-Encoding: identity
Host: hypem.com
Connection: close
User-Agent: Python-urllib/2.7
~Chrome Browser~
Host: hypem.com
Connection: keep-alive
Referer: http://hypem.com
X-Prototype-Version: 1.7
X-Requested-With: XMLHttpRequest
User-Agent:
Accept: text/javascript, text/html, application/xml, text/xml, */*
Accept-Encoding: gzip deflate, sdch
Accept-Language: en-US, en; q=0.8
~Cocoa~
Host: hypem.com
Connection: keep-alive
Referer: http://hypem.com
User-Agent:
Accept: */*
Accept-Encoding: identify
Cookie:
At first I thought it must've been the User-Agent however changing that to be that of a web-browser accomplished nothing. The only real difference happened to be the Cookie field. Sure enough, removing the cookie from the Cocoa request caused the site to start returning me valid keys!
Grabbing that Sweet Sweet Music
Attempt 1
Now that I can successfully grab the proper keys, I began to attempt to use NSURLDownload to try to download the song. It did not happen to be so easy however...
Turns out that the site is performing many redirects to soundcloud and although NSURLDownload follows the redirects appropriately, when the time comes to attempt to write the data to disk it fails with an error "NSURLError Domain 3001".
It's a shame I couldn't figure out how to solve the problem for NSURLDownload because it affords me asynchronous downloading and writing to file together. If someone knows what I may be doing incorrectly please let me know.
All that was needed was a switch to NSURLConnection instead (performing it synchronously currently) and I was able to grab the data!
//lets choose where we'd like to store our song
NSURL * directory = [self getDownloadDirectory];
//lets choose the savingURL to be the songname at the chosen directory
NSURL * savingURL = [NSURL URLWithString:[song mp3Name] relativeToURL:directory];
//create the request using: 'http://hypem.com/serve/play/’ + id + ‘/’ + key + ‘.mp3′
NSMutableURLRequest * songRequest = [NSMutableURLRequest requestWithURL:[song songUrl] ];
//grab the data (synchronously currently...)
NSData * songData = [NSURLConnection sendSynchronousRequest:songRequest returningResponse:nil error:nil];
//write to file
if (songData)
{
[songData writeToURL:savingURL atomically:YES];
}
Successs! I was properly creating a file to disk, not getting any errors...however the file size was 0!
Redo Attempt 1
I got to the point where I can make the GET request for the song however there was one final piece of the puzzle missing. Ultimately, I had solved the similar problem in my python script. I had to set the Cookie in the download GET request to the cookie received from the response of the initial request of grabbing the HTML file.
Set-Cookie: "AUTH=03%3A430aaa119c1852924ef832bbaf5fa989%3A1301502627%3A2170648814%3AON-CA; expires=Fri, 26-Mar-2027 16:30:27 GMT; path=/; domain=hypem.com"
Setting the Cookie value to the one initially retrieved solved the problem and the music started to flow!
The missing line to the code above is the following right after the request is created:
[songRequest setValue:[hypeController cookie] forHTTPHeaderField:@"Cookie"];
Cute Easter Egg
What was amusing was that while inspecting the packets of those sent/received from HypeMachine I noticed an easter egg embedded inside. Here is one of the HTTPHeader fields given in the response packet given back by the site:
X-hacker: Hey, if you're reading this, you should drop us an email at hypem.com/contact, maybe we can work together!rn
