f

unk rock


Google, iPhone, and Thinking Like a Programmer

November 20th, 2008

Yesterday, John Gruber declared that Google Mobile uses private iPhone APIs. Since he wrote it, it must be true, which is why TechCrunch picked up on the article this morning. Nevermind the fact that the only evidence offered by Gruber was that “there is no public API in the iPhone SDK for using the proximity sensor in this way.”

The critical mistake Gruber makes is assuming that because there’s no documented method for trivially utilizing the proximity sensor, there must be no possible way to do it. This is a severe lack of imagination.

Programming is an exercise in creative thinking, especially when working within the context of specific platform. There are always things you, the developer, want to do but don’t seem to be able to. This has always been and will always be true. It’s the reason why plenty of desktop Mac apps use private Apple APIs, and it’s the reason why plenty of iPhone apps on the store are using undocumented APIs.

The essential question, then, is can what Google is doing be done without using undocumented APIs on the phone? I think it can. I spent an hour this morning trying to prove it, and came up with this iPhone app.

To be fair, this doesn’t work as well as Google’s version, or exactly the same way. For one, I didn’t bother to program in the proximity requirement for the trigger. This is well documented, so anyone could easily add it to this project. The other way in which my method underperforms Google’s is that the phone actually has to touch your ear before it will trigger. Google’s just needs to come close to your ear. But, the code does use the proximity sensor, and it only uses the available API method for that sensor, which gives you the ability to turn it on and off. The code uses no undocumented APIs.

How does it work? As I mentioned, it requires you actually touch the phone to your ear. Once you do, the application receives a touchBegan event, and turns on the proximity sensor. It also makes a note that the app is in the middle of a touch sequence, and finally it fires off a timer which makes this trick work. Once the proximity sensor is turned on, it is immediately engaged because the phone is so close to your face. When the proximity sensor is engaged Cocoa turns off the touch sensor, but it doesn’t send a touchEnded message to the application. Thanks to that timer we fired, we can poll for the current number of touches on the screen. If it drops to zero, but we never saw the touchEnded message, we know we’ve triggered the proximity sensor. The project I’ve included will turn the screen yellow and play a little sound, just like the Google application. Here’s all the relevant code:

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
{
    [self setBackgroundColor:[UIColor redColor]];
    [UIApplication sharedApplication].proximitySensingEnabled = YES;
    
    _inLiveTouch = YES;
    
    [self performSelector:@selector(checkTouches:) withObject:event afterDelay:0];
}

- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event
{
    [self setBackgroundColor:[UIColor greenColor]];
    
    _inLiveTouch = NO;
    [UIApplication sharedApplication].proximitySensingEnabled = NO;
}

- (void)checkTouches:(UIEvent *)event
{   
    if (_inLiveTouch && [[event allTouches] count] == 0)
    {
        [self setBackgroundColor:[UIColor yellowColor]];
        AudioServicesPlaySystemSound(_sound);
        return;
    }

    if (_inLiveTouch)
        [self performSelector:@selector(checkTouches:) withObject:event afterDelay:0.5];
}

An hour of programming, and three methods get us something relatively close to what Google is doing. I’m far from an expert in the iPhone SDK, but it only took a little bit of imagination to come up with this idea. Google has perhaps a dozen or more employees working full time on the iPhone. It’s not much of a leap to believe they figured out an even smarter trick than mine to accomplish what they wanted to do. Gruber did mention later that at least one other app is doing something similar without using the proximity sensor, but was quick to point out how inferior it was to the proximity based approach.

Now, as it turns out, Google probably is using private APIs to do what they’re doing. Erica Sadun, over at Ars Technica, made a more convincing case.

The ultimate question then, is does it matter? The reality of the situation is that there are a lot of apps doing this that have already shipped. It isn’t just Google, and it isn’t just big companies, it’s everybody. Either Apple knows this and isn’t doing anything about it, or they don’t know. Either way points to indifference. If they know, it’s obvious how not taking action is indifferent towards things, but what about if they don’t know. In that case, it means the review process they’ve built is not designed to catch these cases. To be sure, the dynamic nature of Objective-C would make it difficult to catch all instances of runtime tomfoolery, but it’s not impossible to do so, nor would it be particularly burdensome to catch most of the cases. As it currently works, I believe they are only checking to see which frameworks you link against statically, and denying anyone who includes private frameworks in that list.

Even if Apple was deeply concerned that people were using undocumented APIs, would it be a big deal if Google was granted an exception? Gruber thinks so:

Third-party iPhone development is purportedly a level playing field. If regular developers are forced to play by the rules, but Google is allowed to use private APIs just because they’re Google, the system is rigged.

Rigged? Who said the iPhone SDK was a level playing field in the first place? Apple has never been a company that stands up for level playing fields, nor should they have to be. Every business makes business deals, partnerships, exclusive rights agreements — this is common business practice, at Apple, Google, and everywhere else in the business world. Why should the iPhone be any different? If you had billions of dollars in the bank, Apple would talk to you too.

This isn’t only realistic, it’s also reasonable. One of the reasons to prevent people from using undocumented APIs is not wanting applications to break between upgrades. If Apple knows every single application that will break, because it has an arrangement with the developer, it can make sure to correct the problem in advance. This obviously doesn’t scale to every developer, but it makes perfect sense to do so on a limited case by case basis. In fact, this is how Apple already behaves in the desktop world. Wil Shipley gets his bugs fixed faster than Joe the Indy Developer. Why? Because his app is popular, and because he has connections at Apple.

TechCrunch claimed in their article this morning that “Apple wouldn’t allow Google to use an unsupported call. It’s not in their DNA.” If anything, Apple is more likely to help Google than any other company. After all, Google provided Apple with a special version of maps for the iPhone application, and now the company refuses to license Google Maps to any other iPhone developer, even the ones willing to pay the outrageous sums of money a license costs. Google gave Apple a virtual maps monopoly for the iPhone, the least Apple can do is let them use an undocumented method here and there.

Ross at 11:37 am | Posted in Apple, Projects, Technology, Web | Comments (3)

iPhone Touch Events in JavaScript

August 19th, 2008

Yesterday I gave a presentation at the San Francisco JavaScript Meetup about the new JavaScript touch events API in iPhone 2.0. I thought I’d share the slides, in case people were interested in viewing them after the fact.

I owe a great deal of thanks to Neil Roberts from SitePen, who’s post on the topic was my main source of information. My co-founder Tom Robinson was also helpful, thanks to his iPhone light-table.

One of the interesting things I was able to talk about was reusing existing mouse based libraries with the iPhone by using touch events to simulate mouse events. The basic idea behind it was to address the fact that mouse events don’t work particularly well on the phone. It’s unpredictable when you’ll get them, if at all (you won’t if you attach only to document, for example). Mousemove in particular doesn’t work in any meaningful way.

In contrast, the touch events are completely predictable and reliable (excepting the existing bugs). We can use these events to simulate the existing mouse events with a fair amount of accuracy. In particular, when one finger is down on the screen, touchstart, touchend, and touchmove correspond nicely with mousedown, mouseup, and mousemove. There’s some difference in the way mousemove works, since we can’t get mousemove events when the finger is not touching the screen, but this is an acceptable trade off. It doesn’t make sense on the iPhone anyway because there’s no persistent cursor.

I took this jQuery based drag and drop example to see if I could get it working on the iPhone without modification. Here’s the working iPhone demo. The only existing code that I modified was some CSS to make the page fit nicely on the iPhone screen, and a viewport meta tag. The added code to get drag and drop working is really simple:

function touchHandler(event)
{
    var touches = event.changedTouches,
        first = touches[0],
        type = “”;
    
    switch(event.type)
    {
        case “touchstart”: type = “mousedown”; break;
        case “touchmove”:  type=“mousemove”; break;        
        case “touchend”:   type=“mouseup”; break;
        default: return;
    }
        
    //initMouseEvent(type, canBubble, cancelable, view, clickCount,
    //           screenX, screenY, clientX, clientY, ctrlKey,
    //           altKey, shiftKey, metaKey, button, relatedTarget);
    
    var simulatedEvent = document.createEvent(“MouseEvent”);
    simulatedEvent.initMouseEvent(type, true, true, window, 1,
                              first.screenX, first.screenY,
                              first.clientX, first.clientY, false,
                              false, false, false, 0/*left*/, null);
                                                                            
    first.target.dispatchEvent(simulatedEvent);
    event.preventDefault();
}

function init()
{
    document.addEventListener(“touchstart”, touchHandler, true);
    document.addEventListener(“touchmove”, touchHandler, true);
    document.addEventListener(“touchend”, touchHandler, true);
    document.addEventListener(“touchcancel”, touchHandler, true);    
}

I’ve captured the touch events and then manually fired my own mouse events to match. Although the code isn’t particularly general purpose as is, it should be trivial to adapt to most existing drag and drop libraries, and probably most existing mouse event code. Hopefully this idea will come in handy to people developing web applications for the iPhone.

Update: In posting this, I noticed that calling preventDefault on all touch events will prevent links from working properly. The main reason to call preventDefault at all is to stop the phone from scrolling, and you can do that by calling it only on the touchmove callback. The only downside to doing it this way is that the iPhone will sometimes display its hover popup over the drag origin. If I discover a way to prevent that, I’ll update this post.

Second Update: I’ve found the CSS property to turn off the callout, “-webkit-touch-callout”. You can read about it in Apple’s documentation.

Ross at 10:44 am | Posted in Apple, Projects, Technology, Web | Comments (7)

Safari’s Web Inspector

April 21st, 2008

The WebKit team announced the list of participants they’ll be working with on this year’s Google Summer of Code today. Of particular interest to me was this applicant. Anyone who does serious web work can attest to the difficulty of debugging anything in Safari, meaning that most web developers have pretty much given up and develop in Firefox full time. This is such a shame, since Safari is almost universally a better browser — it’s faster, it supports more of the things I care about, and for all the reasons Gruber listed (and more), it’s the browser I use personally.

The thing is, the WebKit team has known for some time that they have serious issues with debugging. In fact, I’ve been told better debugging support is the number one requested feature. We saw some attention paid to the issue with Drosera, but it turned out to be nothing more than lip service. Our code brings Drosera to a halt in seconds, as do most of the complicated sites we’ve tried it on. Safari desperately needs a profiler, stack traces, and someday step through debugging. Even beyond advanced tools like what Drosera promised and what Firebug has, WebKit fails on some of the most basic levels. This bug is damning. It should be a P1. Any callback function (read: any AJAX request, event handler, etc.) fails to report the error it encounters. Almost every error in our code base is reported as “undefined” by Safari.

Keishi, you’ve got an important job ahead of you. You’ve been tasked with doing what might be the most important thing the WebKit project has done in a long time. Safari has been kicking ass in pretty much every other area, but it’s ignored by web developers precisely because it’s so cruel towards them. I wish you the best of luck in doing this project, and if you’re looking for feature requests, I’ve got at least a million. At the top of my list? Solve the most important limitation of Firebug — not properly recognizing anonymous (eval’d) code. I have high hopes for what this could be by the end of summer, and I’ll definitely be following the progress.

Ross at 1:12 pm | Posted in Apple, Technology, Web | Comments (1)