Category Archives: Web Development

The Power of Google Fiber

Great post online at RWW called “The Genius of Google Fiber” and I agree on many fronts but wanted to elaborate on the advertising angel.

It is clear by now that with everyone who uses the internet, with every minute you use it more, Google wins, and they have the data to prove it. When a big telco asks if you want to have Gbit Internet for $2000 a month, most will say no, but for $50… sure. That is not something that the big telcos want though.

But let’s do a calculation. You are online 24 hours a week, 100 hours a month on average. 21% of that time is spent searching, of which most if not all is Google anyway. This means that you are searching for 20 hours a month. In 2011, Google had a billion unique users a month, so let’s presume they have 2 billion now. They also have 5.6 billion adimpressions per day on search, on average i’d guess 2 ad impressions per search, so 90 billion searches a month. Each user does 45 searches per month. On average 65% of the clicks are paid and I presume that there are three clicks per search on average. Means 135 clicks per month with 90 of those being paid, at $0.35 on average. That’s something around $30 per month. That’s actually close to the $43 per user cited on Motley Fool.

With Google Fiber they have people watch movies online, be faster, be able to do more in the same time, and so on. Can you get that time from an average of 100 hours per month, to 150 hours per month, which is just 5 hours and includes TV. That would mean an extra $20 that Google will make. Remember this is an average, you just get a lot of people from a few minutes a day. And it is a lot more complicated than that.

But, what becomes visible is that Google will possibly make something like $20 per user in Ads without getting anything for the pipe.

This makes it very very interesting for Google and is a revenue stream that the telcos cannot match. And it does not factor in the lockin value, and other things.

The telcos better beware.

(P.S.: and this is just a thought experiment and probably wrong in different points but also probably not too far off ;) )

The Hubbub about PubSubHubbub

Ok, you have to love the name. Pubsubhubbub is simply the best name for a technology for ages, period. And it is currently pitted as the better step towards a Google Wave like future than Google Wave itself.

So what is PubSubHubbub:

A simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom (and RSS).

In longer terms, it is part of a system that removes the need for Google Reader to query my RSS feed every few hours to see if there is something new, instead allowing for Google Reader to be notified when something changes. The argument is that it gives us the live web and it decreases resource need. For a better into check out This Week in Google Episode 2.

I really like it and so does Anil Dash as visible in his post The Web Way vs. The Wave Way.

There is just one thing that makes me wonder. I remember a long time ago when MovableType was it, with everyone arguing that it was better to have the blogging platform publish static HTML files when you published a blog post as you are publishing so few of them in relation to the traffic that the resources are better spent there. But servers got more powerful and WordPress changed this around to say that you want easy and fast publishing, the possibility to change your design around, the switch things, which makes it better to have the publishing be dynamic. This is what I see again here. We had centralized systems and only a few of them but slowly we are getting more and we want to move around. This changes how the content should be put together. We have friends all over the place and hence on request we do not want them to come from one source but from several, which is starting to become possible due to server power and general infrastructure items like Pubsubhubbub. It’s push due to the underlying changes in how we interact with each other and what we say is important.

We somehow want near-real-time communications. The problem I see though with Pubsubhubbub is how the technology scales if you have for example 1 million friends on twitter. With a centralized system this is relatively easy. If each of your posts needs to be published to 300.000 hubs, this starts to become a pain. But with a polling mechanism it would not be live. How much resources are we will to spend for live, and who will pay the bill?

Moved Blog to Squarespace

Good day everyone and welcome on the new home of my blog! I moved everything over to Squarespace after being brainwashed over weeks by TWiT. :)

I have to admit that the system is really nice, even though I still have a small problem with importing my old entries into the blog (new lines get lost but can be revived through a short edit). This is supposed to be fixed in the next two weeks. Customer care by them has been amazing with questions answered mostly within minutes. Another item is that I still need to use friendfeed as a lifestream system for the homepage, but it seems that lifestreaming is coming in due time, currently in beta testing.

I just wanted to get rid of managing my own server even though I loose the opportunity to play with a lot of fun stuff … which might be good anyway, you know time and all. This new site will hold an aggregated stream on the frontpage (once out of beta or me in the beta ;) ), my blog, a short about me, contact information as well as a private section for family which is very easy to set up on Squarespace. This actually covers all I need from my system.

The only problem was changes in URL structure but I hacked together a small PHP Script that is now handing over permanent redirects to search engines to the new urls which was easy enough.

Looking forward to more interesting things from Squarespace (i.e. iPhone app :) ) and talk to you soon.

Review of Making Things Happen by Scott Berkun

Making Things Happen is one of the best project management books I have read in a long term, period. You can stop reading now and just buy the book if you want. Scott Berkun also has a very good Blog to follow. Otherwise, continue for a summary of the book, which is unique to me though. This is something that Scott makes very clear, in that project management is different for everyone and should be.

He starts out by saying that we need to learn from the past, which Boeing does with something they call a Black Book. This learning from the past also means from different professions. If you think you have difficult project management, watch a kitchen work in full forward press and you will think differently for example.

I liked his starting point that a project manager might actually not be needed as a dedicated position if you can distribute the responsibilities to different people on the team, which we are doing here actually. But you need somebody to lead the bug triage and who is looking at the schedule which we will enforce now. Another important role at Microsoft is actually the program manager who is kind of doing everything that nobody else does, being a kind of connection person between everybody from tech and business sides.

But already a PM as such has two hearts in that she has to have an ego but not an ego, an autocrat that wants to delegate and tolerance of ambiguity in the pursuit of perfection … they are leaders and mangers.

In the end, and I loved this, there are three things in a project that you need to focus on: a goal, a pile of work and a bunch of people. And this is what the book is about. How to create goals, manage piles of work, and work with people.

In that, for the people, well defined roles might help but should never become the goal. Remember always: What is your goal? Is everything you do focussed on that goal? Run, stumble, learn, adapt, run …

So what are leaders and managers? “Leaders and Manager are hired to amplify the value of everyone around them.” Great quote.

So what is the truth about schedules? They are a psychological forcing function and lead to three purposes: a commitment, the forcing function showing part of a whole, and to track progress.

Then there is a rules of thirds, which will directly adapt to have even closer monitoring of goals: design, implement, test … all take up a third of the time of a project, whereas test includes verifying, analyzing and refining a feature. But remember, in any schedule you are trying to predict the future and this is always rather difficult. A schedule is always a set of probabilities, which means you should also define what your schedule means. Is it a guess (40% accuracy), a good estimate (70%) or a thorough analysis (90%).

So how does slippage in a schedule occur. You can ask these questions: Are there sick or vacation days; holidays; do we have regular progress reports; is somebody watching the schedule; is there ownership of the schedule in the team; are new feature requests blocked; is there a probability in the estimates; are there good specs.

If there is a weak or actually no vision document (high level goals) x poor or no specs x poor or aggressive estimates x no budget for integration = a prayer of a schedule

For requirements you will have three views: business, technology, customers and customers always has the lowest focus which should not be like that. Your Work Breakdown Structure (WBS) should come from Specs, which comes from the Vision documents who come from very high level requirements. In our case we get requests from internal and external customers, who come into a prioritized list, which move into sprints with a clear vision, closer but short specs built into a word breakdown structure per responsible.

But what is a good vision? A good vision should simplify, making the group rules clear, should be SMART, consolidated, inspirational and memorable.

Here thinking inside/outside/under the box is unimportant, keeping the constraints of the goals of the project in mind is. You should start your design from the customer perspective and mull over it with a diverse set of people to get lots of different experiences involved. The next problem is choosing a design and here you need to make sure you let the team know that it is time to converge after diverging for ideas.

Specs as such should be written by one person and the more iterative a development process is the less specs are needed, which kind of solved my problem of wondering what I should spec in great detail in 2 week sprints ;)

The important thing is that specs do three things: the right thing is built, the schedule includes a planning phase, and the specs allow for deep review. This is actually one reason we often build designs in the sprint before they are implemented, to have a bigger feedback cycle.

Then comes the decision part: He lists 7 points to think about that normally you wouldn’t go through. Hi first point is the typical 5 Whys and then of course there are general things like impact, experience, experts opinion and is there approval needed? Also make sure you only make decisions on things you need to make decisions on. Then go to the heart of the problem through e.g. Occam’s Razor and take your time to reflect. Option lists might be important here.

Remember, and this is why we have short sprints, communication is still the biggest problem, see Tower of Babel as a story.

Then comes the people problem. How to annoy them: assume they are an idiot, don’t trust them, waste their time, manage without respect, make them listen to stupid things.

In terms of projects always see if they: accelerate progress, prevent problems, make important actions visible and measurable, include a process for changing and eliminating the process, people impacted are in favor of them.

You can look at a function if you want: design time + learn time + (actual time of the process versus the time it took before * times used) needs to be smaller than (failure costs * failure probability * time this happens over the time of the use of the process). In the end though the time of using the process should be lower than the time you needed before.

Quote: Mark Twain (corrected from Pascal from @taospace): If I had more time, I’d write a shorter letter.

Going on with people he quotes Virginia Satir in saying that our response to a feeling is often not the first feeling but a second one. Sounds confusing as such, but think of me telling you that you smell. That should as such make you sad that you smell, but because I made you sad you feel angry towards me. Keep that in mind at all times.

In the end, leadership is based on trust, and therefore a leaders behavior should be predictable. Do what you say, say what you mean, admit when you are wrong.

In terms of why lists are important, he says that you need something simple to know that the most important thing gets done first. I agree here, but we keep our priorities a lot simpler than what he has listed. But you need to learn to say no and I agree here, while no is often something that is moved to a “To be Speced” or “RFC” sprint that is always open. Say no because: it doesn’t fit the prios, only if we have time, only if X happens, next release and … most important … never. ever. really. Always keep in mind the critical path.

There is lots more stuff in there but I’ll end at that. No on to building our triage process ;)

Getting that Damn Address Book in Sync

This will likely be an ongoing problem I have to say but I really hope somebody will start building something that works. The even bigger problem is that the real problem (understand by now that it’s a problem? ;) ) that likely the players that hold (part of) my address book will need to support any move for a sync.

But let’s move one step back. I tried a beta service called Soocial and didn’t back up my address book. Stupid me, agreed, happens a lot of times recently because I have my head full with other stuff and I expect things to work (it’s taken care of now). It synced with my local address book (close to a 1000 contacts), then with GMail (up to 1600) and suddenly it was going up, doubling contacts left and right and I was up to close to 3000. All that happened in the background. The service provides backup possibilities meaning it should have had a backup from my first sync but that backup was empty. And it got worse and worse and suddenly all was gone. Thankfully I have a sync with GMail and long ago decided to try to bring a large part of my contacts into Xing.

So one export from GMail, fill up with Xing data, and I am moving in the right direction again. I now set up an account with Plaxo, after I made them delete my last login. It is amazing to see what kind of diverse group of people are in Plaxo I have to say and I am now using it as a focussed system to sync address book data. I would love it if they would hook up with Xing.

Strangely enough I did not find any way to export my previous address book uploads to LinkedIn, which would also have been a wonderful way to get back that data.

I really would like somebody to take care of all of this. There are different backup solutions, and I might try Soocial again at a later date, who are now probably the closest of the bunch I know to solve the problem. Plaxo for example only has one way Sync into GMail.

What I would like is something that really syncs what I have. I mean lots of people have a photo inside Facebook, Xing, LinkedIn, Plaxo, Twitter or Noserub or whatever so why do I need to add them to my address book damn it! So this would be another interesting project for all of you out there. A real sync of my personal connections. Go cracking. Thank you. :) Suggestions of course always welcome.

Self Generated Content Infrastructure

We have loads of user generated content out there and I like it. The main thing behind getting into blogging years ago was my own personal remberall. I blogged the summaries from my MBA for example. This is also true for del.icio.us, even though I haven’t found the perfect bookmarking tool yet. Furl was actually getting close now thinking about it but it somehow stopped being used. Not sure why.

Now I am starting to become a fan of Brightkite, am still an avid user of Flickr, still love Twitter, have my own Tumblr like blog (thinking about moving to Tumblr from Chyrp hating the not possible import feature). I am also on facebook which aggregated my twitter stream and facebook aggregates it all, actually double aggregating some stuff because it’s autotumbled.

The problem I really have is that I want all that. I want it in one system where I can add private feeds and can search for it. I want it so it integrates all tags. If I look for tag “advertising” it should list all stuff for this tag, be it del.icio.us, tumblr, or wordpress. I want it to understand my exact location based on using brightkite and other service while not having to publish my location to flickr and other. This system also need to accept imports from all those platforms as RSS feeds are not going back far enough.

Friendfeed is not bad already but I actually want Friendfeed to store all the content of the stuff that comes in from RSS feeds because I want it included in the searches. I don’t want them to show it to me, I just want it returned in searches damn it, so it’s not about duplicate content or something. I often do not have the word I want to find in the title of the post, if I even remember the title of the post.

This is actually what lifestreaming is about but it’s more of a personal lifestreaming thing. I do not care about sharing it with the world. I can, but it is too much for most people anyway. When I look back in my timeline I would love it tightly integrated with my current location at that time without having to tell that to the world. I want public and private photos.

I am inches away to starting an open source project.

Running a Startup on Agile

I have this one sitting in my “To Blog” bookmarks list for ages now, so here it goes. It is actually inspired by a great post by Eric Ries entitled “A new version of the Joel Test“. The original Joel Test was written in 2000 and Eric tried to update it a little bit based on his experience in software development in agile teams. Same as him I start with Joel’s list.

  1. Do you use source control? Yes, essential and check.
  2. Can you make a build in one step? We have continuous integration, so check.
  3. Do you make daily builds? Well, building continuously would qualify I think. Ok, we could always do with more automated test but we are doing ok.
  4. Do you have a bug database? Sure, check.
  5. Do you fix bugs before writing code? There I am with Eric, yes we try to, but bug is not bug. There are some things we accept to fix later. But we do have an ASAP list that gets precendence over anything else.
  6. Do you have an up-to-date schedule? Here I again agree with Eric, damn I agree aa lot with him, but that’s the point. What we learned is that not the schedule is important, but keeping sprints short. As soon as a sprint goes over more than 2 weeks we have a problem of too big a backlog for the next one, too long testing, too much waiting, too hard to test features and the like. Our progress, which is what it is about, is better with shorter sprints, which take less planning and can actually be planned on the spot.
  7. Do you have a spec? Eric rephrases that into “does the team have a clear objective?” Yes, this is what is important. We try to have a 1-3 sentence goal for each sprint. Everything is aligned to that. There are then cross functional teams that build the spec. In the first meeting we create scribbles, get the features down, try to make the interactions clear, and go from there. This first meeting is very important though and we have failed in this before because you need the cross functional team there and you need to discuss things to the end and fight it out and the team needs to stay fixed afterwards. This is sometimes hard, but very important.
  8. Do programmers have quiet working conditions? It’s not one size fits all. Two of our devs moved out of each others office because one likes to hear to music and one likes it quiet. Some want company, some don’t. You need to enable that.
  9. Do you use the best tools money can buy? We try to and this should probably be taking up a bit more, but hey, we are a startup that watches the money.
  10. Do you have testers? Everybody writes tests first and then codes, as best possible. Yes, we have somebody responsible that makes sure everybody does that, but developer is developer. This is still to be decided in my mind. After a certain team size you probably need someone that takes care of the infrastructure, enabling automated testing by developers, and builds new test cases and handles the release process. But for small teams, it’s a team responsibility.
  11. Do new candidates write code during their interview? We didn’t do that yet, but more or less knew the people we hired, so it wasn’t so necessary. What we would be doing more in the future to give a new developer full responsibility for a new feature that is laid out so that it interacts with the entire platform. Then let everything be discussed in the weekly code review.
  12. Do you do hallway usability testing? Majorly lacking here, sorry. We are trying to keep iterations short but will probably have to do some learning in this field.

Next up are some of his suggestions. Yes, we do work in small batches and through daily scrums we know who is working on what and hence there are little conflicts at checking. He also talks about practicing the five Whys which I remember from the MBA. It’s a great idea. Why is it a great idea? Because it allows you to think up till the root cause? Why does that help? Because you don’t fix something that does not need fixing but what really went wrong. Why? Because after the first idea you ask why again and go on from there… get the point? 5 Whys. It’s a great system and oh so simple.
He then goes on to talk about the difference between defects and polish. This is very important to understand. We build a feature, often taking the easy way for the interface (after we learned building it complicated first is nuts ;) ) and how it works but making sure there  are no bugs, that it works, and does so reliably. From there we can build polish if need be. This is a subtle difference, but an important one.
His last point is probably the most important one: Does everyone (he names programmers but that’s not enough) understand how what they are doing relates to the company strategy, well being, vision, day to day business? This is where the cross functional teams help. It helps programmers understand what people do. A good developer will go nuts finding out that somebody sits there doing something in the interface for 2 hours every day when the dev can whip up something simple and get that down to 5 minutes. But you need to have communication.
That is actually one thing that I would like to add. The most important thing, and the reason for being of Agile, is that you have communication. The entire point of now doing waterfall development is that you know that the biggest problem in software development is that people speak different languages. Every party will need to try to work out what the other party really means. That is hard, and painful sometimes but only understanding will lead to good code.

Yahoo! Opens up Search – Some ideas

Now that is interesting news: BOSS – The Next Step in our Open Search Ecosystem. Techcrunch as a positive note and RWW thinks it’s exciting news and I have to agree. Of course as both also note there are limitations to all of this but it is a good first step.
In short, you will have full API access without rate limits to query the Yahoo! Search Backend, getting back results based on your query as for example JSON, being able to do with it whatever you like. In our case at Ormigo we might want to do searches for related pages within Ormigo, enhancing the results based on internal data we have and adding a few external links based on where they came in in the results.
Check out Me.dium for example to see one company already using the system. Actually a very nice idea to use my and my friends surfing behaviour to give me the right results. Sadly I don’t see all my friends installing the browser bar so it is out for me.
In any case this will drive innovation in the space and I am looking forward to see what comes out of it. In the long run you will be able to pay CPM based or you will have to run their ads next to the search results, which makes it a little bit less altruistic, but that’s ok. For now, all is fair game, so go play.

Google Gears attacking the Operating System

This is really as far as it goes. Google Gears, or now Gears, as it de-branded, makes your browser in conjunction with a web site, into an offline application, or simply an application with local components. Because it started out by simply giving your offline access to GMail or Google Reader, and now the folks behind WordPress have announced that WordPress 2.6 will have Gears integrated to speed up the Admin Interface. Matt actually adds a few points here, among others linking to the Performance Page on the Yahoo! Developer Network saying that something like 85% of the performance is in the frontend, meaning it is felt performance. This brings him on to Google Gears and how local servers will make CDNs obsolete.
While I do believe that there is a lot of power in Google Gears, I am not seeing yet where they will make CDNs obsolete. There is lots of stuff I do not cache or cannot cache in the browser. Above that, I know of at least one person that will scream up when you mention something like Google Gears or AIR. The thing is that these are systems that give Web Apps very deep access into your system and we just don’t have the mindset yet to really secure web applications. A simple link sent to you, might result in something loading in some un-secure AIR or Gears app and destroy your computer. This is a risk we have to deal with somehow, but I am pretty sure that is on the mind of the Google engineers and the others that are working on Gears.
I do fully believe Matt though that the line between servers and application is shifting in that you will less and less worry about your server infrastructure. I had a longer talk about this with the CEO from Globalways, who are running a Xen virtualisation Platform.
But back to Gears, Techcrunch has a lot more interesting points, e.g. that Gears was implemented by MySpace for their messsaging.

Today Gears supports a whole host of new features, some that it has in common with the other next generation web API efforts from Microsoft and Adobe while others are a result of their own innovation. Function calls available to developers include background processes (no more hourglass), client-side image manipulation, location-awareness, better file uploading and a local database inside the browser.

It’s really getting interesting here in that you might want to give your users the option to simply download Gears and use it and suddenly can write a lot more powerful web applications, with less servers. Looking forward to trying out the new WordPress to see how it works with Gears.

Amazon EC2 gets Static IPs, Availability Zones and more

How I love Amazon AWS. In October there were already hints about Static IPs and now they are here. Check out the Amazon Web Services Blog post about all of the new stuff.

First of all, we now have what they call Elastic IP Addresses and the system is very cool. You get up to 5 IPs to start with. You get one via an API call to AllocateAddress, which allocates you one fixed IP that then belongs to you. Without you using it you pay 1 cent per hour. But you can then do an AssociateAddress and it is attached to a Server and becomes free, meaning you no longer pay for its usage. You can then DisassociateAddress and ReleaseAddress if you do not plan to use it at all.

Then couple that with Availability Zones, which are zones in their Network Infrastructure that are insulated from each other so that if one zone goes down, another does not (in theory, there might always be odd cases, chance if you want ;) ).

This really means you can do more for a high availability solution with Amazon AWS and if they now start a NOC in Germany, I will possibly never do my own Server again. But they are not here yet so we are just using it for parts of our system, and I am taking a look at Globalways.

But again, congratulations to the entire Team behind Amazon EC2 for pulling this one of. Thank you. No more DynDNS for our Ad Server :)

Follow

Get every new post delivered to your Inbox.

Join 1,881 other followers

%d bloggers like this: