If I think back to 2006/2007 I was very happy with how things were shaking out. The web was on the upswing and we were moving away from native applications. All of the great things about the web–its ubiquity, its freedom, its openness–were being harnessed to create native-like experiences that, I hoped at the time, would see us all do away with native apps. At the time, there wasn’t much (I thought) that web apps couldn’t do that we needed native apps for. How wrong I was. With the introduction of the iPhone, and subsequent smartphone releases, we’ve seen a huge shift back to native applications. Part of that is performance, right now native apps just feel better than mobile web apps, but it also came about because of just how many things native mobile apps could do. Geolocation, accelerometers, contact info–the smartphone showed how many things “apps” needed access to and for the most part those features have been exclusive to native applications. So just as the web was starting to really take off, we’ve slid back into native application territory.

It bummed me out, and still does. I thought AIR was an okay solution to the problem, but by the time AIR came around it was pretty clear that “the web” had come to mean HTML/JS, and I’m fine with that. So as PhoneGap started getting traction, and then Adobe took a major interest in the project, I was excited about the prospect of working on it as an Adobe evangelist, and more importantly, working with the teams behind it to see what else they had up their sleeves as the web moves forward.

But another side benefit is that it’s put me on what I’d consider the “right” side of the web argument. Two things got me thinking more about this. One was a very good post by a VC named Mark Suster, who while not telling his companies to focus exclusively on the web, has told them to make it a big part of what they do. So many startups nowadays are thinking completely mobile-first while ignoring the web, I’d say at their peril. When I think of my own usage, I’m still using a lot of native apps (for reasons I can’t quite figure out) but the ones I enjoy most have a web component that is well done and part of the overall experience. Yelp comes to mind. Another is Untappd. I can do almost anything I need to on the website version of Untappd so it’s not as though I’m getting a watered down experience. It more easily lets me move between contexts and devices while still using the service. I contrast that to something like Foursquare or Path. Both are mobile-centric, and with Foursquare I can’t check in because it wants to be sure that I am where I say I am (using GPS) but it still makes the web side of it less useful for me. Path is unusable when you’re not on a mobile device as far as I can tell. When I log in all I get is “download the app”. Which I hate. Mobile is clearly important but the web can’t be ignored.

If this is what I see when I log into your site, you're too mobile-first.

The second thing that got me thinking more about this was something Brian Leroux pointed to on the Cordova mailing list, a post by Tim Berners-Lee about Installable Web Apps. This is a model I would love to see take hold. As Tim notes, there are a few things that users need to have when they’re installing web apps, and some trust/permissions issues that need to be figured out. Right now, I think PhoneGap is closest to this model, but a huge, huge, part of me wishes PhoneGap didn’t need to exist. If we could somehow skip the native shim and just take for granted that every platform supported, and at its core used, installable web apps. Maybe something like the WebOS model. But we’re not there yet. So for now, I’m glad I get to work with PhoneGap and build apps with web technologies. Eventually though I think PhoneGap can be used as inspiration for installable web apps. This is kind of how the standards world moves, as more and more people adopt something, people find ways to bring that something back into the standards. I think some of PhoneGap’s APIs and methodologies would make a great start at the idea of installable web apps. And I think the guys behind PhoneGap will be at the forefront of making those things happen, which means Adobe is going to be a really cool place to be over the next few years. It feels like there is a lot of potential to change the world and while I miss spending time with Flash, I feel like the HTML/JS/CSS work I’ve been doing and that Adobe is investing in, will make a similar impact on the web down the road.

I’m not sure how helpful this will be for anyone, but during the process of doing a binary WebSocket demo I found myself learning a lot about JavaScript typed arrays and how that translates into binary data. The demo I wrote took Canvas image data and sent it over a binary WebSocket connection. The WebSocket server took that data and sent it out to all of the connected clients, who would then render the Canvas data as a PNG. It’s kind of a niche use case but I wanted to specifically create a binary WebSocket demo. It also was a more efficient way to send image data than doing something like base64 encoding it. First off, the Mozilla documentation was REALLY helpful. Major props to them.

Canvas has a getImageData() method that gives you an ImageData object. Within that ImageData object is a data property, which has the actual array data of the image. Normally I would have been able to stop right there because that would (in theory) have the information I needed. But what I had to get at was the ArrayBuffer. The way the spec has been implemented you can’t do anything with the ArrayBuffer. Instead you have to use an ArrayBufferView, which takes the form of TypedArrays in JavaScript. Luckily, to get the actual buffer you can just call the buffer property on any of the typed arrays and do what you want with it. But Canvas (at least in Chrome) is slightly different. The object you get from the ArrayBuffer of ImageData.data is something called CanvasPixelArray. Currently CanvasPixelArray doesn’t behave like a regular typed array, it looks like it will become a Uint8ClampedArray but the way it works in Chrome right now the CanvasPixelArray doesn’t provide access to a buffer property so you can’t send/access the ArrayBuffer data. Luckily getting that data into a Uint8Array, which you can get the buffer data from is pretty easy:

     imagedata = context.getImageData(0, 0, imagewidth,imageheight);
 
     var canvaspixelarray = imagedata.data;
 
 
     var canvaspixellen = canvaspixelarray.length;
     var bytearray = new Uint8Array(canvaspixellen);
 
     for (var i=0;i<canvaspixellen;++i) {
          bytearray[i] = canvaspixelarray[i];
     }

With that new Uint8Array all that we have to do is grab the buffer property and we can send it across the wire (beyond this post, but I’m planning on writing up the binary WebSocket info).

Now what I wanted to do was take that binary data from my Canvas and render it as a PNG file on the screen. The first step is to use a Canvas object to render out a PNG. But before that we have to get the data into a Canvas. In theory, you should be able to just do what happened above in reverse. But it’s not quite that simple. Once you get back the binary data from somewhere (WebSocket say), you’ve got an ArrayBuffer that you have to deal with. There is a putImageData() that takes an ImageData object, and we can create an ImageData object a few different ways, but you can’t set the data property of it. It’s read only. So we can’t take our data from the ArrayBuffer and just put it into our Canvas. We have to manually loop through the data property and line-by-line change the data.

          var bytearray = new Uint8Array(event.data);
 
 
          var tempcanvas = document.createElement('canvas');
               tempcanvas.height = imageheight;
               tempcanvas.width = imagewidth;
          var tempcontext = tempcanvas.getContext('2d');
 
          var imgdata = tempcontext.getImageData(0,0,imagewidth,imageheight);
 
          var imgdatalen = imgdata.data.length;
 
          for(var i=8;i<imgdatalen;i++)
          {
               imgdata.data[i] = bytearray[i];
          }
 
          tempcontext.putImageData(imgdata,0,0);

If the above isn’t clear, basically I’m just creating a new Uint8Array with the data from the server, then creating the temporary Canvas so I can get image data from it, and when I have that, I’m looping through the data property and inserting my own data from the Uint8Array.

So now we have a Canvas (not being displayed) that has all of the data from our server, so it’s an exact graphical copy of the info we received. Turning that into a PNG is actually pretty easy because HTML is awesome. Canvas has a toDataURL() method that will take whatever is in the Canvas and create a string that can be put into the src property of an image. Then putting that image somewhere on the DOM will display the data as an image.

          var img = document.createElement('img');
               img.height = imageheight;
               img.width = imagewidth;
               img.src = tempcanvas.toDataURL();

Now that I’ve gotten my head around binary data a lot more, I’m kind of excited about JavaScript typed arrays. Looking through the list it looks like the typed arrays will help a lot with byte manipulation because of the different types. It also looks like they’re pretty fast (at least the fastest option at the time of that post).

I enjoyed this post by Grant Skinner that walks through his view of the evolution of technology and where/how/when tooling starts to come in. Adobe makes tools for web professionals. That’s what we’ve always done and that’s what we’ll do for a long time. You could even paint a broader brush that we make tools for creative people to share their creations. Watching our own evolution over the past year or so with regards to HTML tooling has been very interesting. We got some flack for not moving in earlier, but as Grant rightly points out, tools are a major investment and only once you have stability can you make that investment. It was never a matter of momentum around HTML or a focus on Flash, it was just the fact that things weren’t quite ready for tools.

In fact, I’d argue they still aren’t. But we’ve taken that as something that comes with the web. It’s always evolving, always moving, and while things will start to coalesce more and more, in the end, you have to get in and be ready to move. That’s kind of the approach we’ve taken with Adobe Edge. We just released Preview 4 of Edge which incorporates a lot of features that people have been asking for. Some of it I’m not even sure if it was on the original roadmap. But the Edge team made a conscious decision to be very agile, to build Edge in such a way that it could incorporate customer feedback quickly, and then getting product management on board to do lots of versions very quickly. I think it’s worked out very well and despite being on the earlier side of Grant’s curves, I think Edge will be a very helpful tool for a lot of people because of it.

Developer tools are a bit of a different story in terms of both ecosystem and readiness in my mind. There isn’t one, big, HTML IDE that people seem to like (akin to Flash Builder, Eclipse or Visual Studio). Instead people seem to be using a lot of different things and experimenting. What actually seems to be most popular right now are the basic text editors like TextMate or Sublime (my favorite). These seem to be focused on helping smart people work faster. Lots of shortcuts, lots of snippets, but not a lot in terms of helping along the learning process. And I think that’s just where we are now as far as HTML/JS/CSS tooling. But I’m excited to see that evolution as well and see what happens when frameworks get a bit more standardized and more general web developers start jumping into JS more and more. Will those people need a more robust HTML/CSS/JS editor that’s still developer centric? And I think the answer is yes, but I think it’s also tough to really see what that would look like until the JS/HTML/CSS stack is a bit more solid. But I’m excited to watch it and find out. And from what I’ve seen of Adobe’s HTML tooling side, we’re taking a good approach and I’m excited to see what people think as the PMs share more and more of it.

Jeff Croft has a nice riff on some of the responses he got when he tweeted about what Steven Witten has done with his site, acko.net. It’s an insanely cool piece of CSS and 3D work and Steven did an informative writeup

It’s fascinating to see some of the anti-Flash attitudes manifest themselves in the new world of a very rich HTML5. And I think it goes to show that a lot of the animosity towards Flash was rooted in a very strict interpretation of what is “good” when it comes to UX and UI. I don’t even think that’s entirely wrong, but I do think that some of it misses the point. As Jeff noted, you have to have experiments like this that show off the technology. This kind of thing inspires people to think beyond the gimmick and potentially implement some of these ideas in a more usable and meaningful way. It also does a disservice to the fact that things like this make the web a more beautiful place. This isn’t “design” in the traditional sense. But it’s still art, and art should be celebrated. Most of the things I’ve seen on Twitter were pretty positive about the site and appreciated it, but the opposite reaction from a lot of the web standardistas was, as Jeff said, disappointing.

My plea for the potential creators of things like this is to not heed the mindset that because you’re using open standards you need to adhere to the unwritten rules of UX/UI design and what’s “good”. The open web is at the beginning of what is going to be an amazingly creative period. With technologies like CSS3 and Canvas, the open web has the building blocks in place to harness a lot of creative energy. That is going to take many forms and some of it won’t seem as useful as other bits. But it’s all part of a process, one that we saw with Flash, that ultimately leads to better ideas, better implementations, and a better web.

I had the chance to present to the Seattle Web App Developers Group (formerly the Flex UG) and here are the slides from that presentation. Thanks a ton to everyone who came out. It was a lot of fun and it was exciting to see so many people interested in PhoneGap.