Thursday, June 25, 2009

Google stalling advancement of the Internet?

This may sound like a far fetched claim aimed only at creating controversy but let’s consider a few facts about Google search and how it is impacting the trends in website design. For the record, I am a big fan of Google and what they do for the web community. Google has envious track record of bringing new technologies for the distributed environments however, their main claim to fame - search engine, is really a backward looking beast that belongs to the previous decade. And because Google search engine dominance and the fact that there is still nothing better around, we may have no other choice but to succumb to its old-fashion “view” of the Internet.

The issue came to my attention when I started searching for information on how to best design a catalogue application for my site, dedicated to maps and travel related information. A clear pattern emerged pretty quickly. Websites with lots of pages and mostly text content are the winners as far as traffic and high Google search ranking goes. These sites may be totally irrelevant for people searching for specific information but that is beyond the point. ”Just get the suckers in” and show them lots of ads to click on (and users are more likely to click on advertisements if they cannot find what they are looking for!). And that is the real purpose of those sites…

It is important to highlight the fact that for majority of websites Google search is a lifeblood without which they cannot exist. Sure, once the sites reach a certain size, or operate in a certain part of the online market (eg. social networks), the importance of search engine traffic diminishes significantly. But for many small to medium size websites search engine traffic constitute a significant proportion of users.

It is therefore very unfortunate that the most popular, mathematical algorithm driven search engines cannot distinguish between sites with lots of irrelevant information and those that provide succinct and highly targeted content presented dynamically on a single page. It is an irony that Google search page does not have much content! It would never be discovered if it was not already popular (it comes up on number 5 position for term “search” after bing, dogpile, AltaVista and Yahoo, yet it’s the most popular search engine in the world).

Google search engine has its roots in very simplistic approach to web design when links were used purely for navigation between pages and sites, and text placed on the page defined its content. This approach is no longer relevant in dynamically created, context driven Web 2.0 world.

I have a real dilemma how to progress with my next project. I want to create a catalogue of free maps on the Internet. My initial prototype proves I can deliver the entire content and search functionality via a single page. Yet, this approach means that Google search will treat the application as of no real significance (being only a single page) and maybe even as a link farm and remove it from search index altogether (since navigation is through dynamically created text links). An alternative is to do it in a “search engine friendly” way… with 1,000’s of pages and lots of text.

If I create the catalogue as a dynamic, single page application no one will ever be able to find it. This in turn will defeat any technical and user convenience related advantages that support choosing this option in the first place. So, because of enormous influence of its search engine in the cyberspace, is it possible that Google is inadvertently stalling the advancement of the Internet after all?

Tuesday, June 23, 2009

The golden rule of online business - traffic, traffic and traffic

Traffic is a cornerstone of every website, whether it is just a personal blog or multi-million dollar corporate portal, or brand building exercise, or else. Even if the site does not carry any advertising, without the traffic there is no reason for “it” to be there in the cyberspace. Unless it is just a fruitless exercise in self indulgence…

For the big end of town traffic means survival or extinction. Not long ago one of Australia’s aspiring real estate sites was pulled down because it “only” attracted 500k visitors a month and there was no chance of it ever being viable. On the other hand, sites that are able to bring more and more users are very attractive for potential investors, even if they are totally unprofitable (see my recent post “Millionaires abound! Business lesson from Facebook”).

Changing trends in user preferences that affect traffic bring havoc to online enterprises. Recent demise of the once all mighty America Online (AOL) is a point in case. The battle between MySpace and Facebook is another example, with forgone conclusion [as my daughter put it, “MySpace is messy and only for small kids”… but “…you must be on Facebook otherwise you don’t exist socially!”]

Even companies that rely on off-line revenue sources but operate primarily in an online environment (eg. realestate.com.au, wotif.com, seek.com.au to name a few locals) are dependant exclusively on traffic. If they were unable to attract visitors their off-line revenue would dry out quickly and share prices would bomb.

For smaller sites, traffic means the difference between being just a time consuming “hobby” (even it the site is created by a corporate entity) or expert and useful source of advice, information, entertainment, online services etc. What’s the point of putting all that effort into maintaining the site if no one is there to see it?

Traffic drives revenue and traffic drives value of online business, or hobby, as was “proven” in my recent post explaining relationship of traffic to advertising revenue and valuation. Therefore, looking after your traffic is the most important aspect of online presence.

Friday, June 19, 2009

Ingenuity of Google Map architecture also its main limitation

When Google Map was first revealed to the world many in GIS industry were stunned how simple to use, fast and good looking it was. Such result was possible only because of ingenuity of Google map creators.


The traditional way to publish maps over the Internet was to use specialised mapping servers to render GIS data into a simple image and then to serve that single image to a browser for display. A new map was generated every time users panned to a different location or changed zoom level (since there is an infinite number of combinations of map extents and zoom levels it was impossible to generate those maps in advance). And those maps were always less than perfect because every display rule had to be predefined for the entire map and for every possible zoom level. In case of very detailed maps the complexity of defining those rules was very often beyond the capacity of map creators.



Google map developers had broken with that tradition. They opted for a solution where maps could be produced in advance and served as small tiles for assembling into one big image at user end. The advantage of this approach is consistency of appearance and graphical quality of the map (which was rare prior to release of Google Map!) and, probably more important, enormous scalability that could be achieved. There is no need for server side processing to generate maps and individual map tiles are much smaller than the whole map presented at the user end, so they are able to be delivered and displayed much faster. The trade off was a big effort up front to generate nice looking maps and the need to fix zoom levels rather than allowing a continuous zoom, as is the case with the traditional approach. It was solution tailored for the web, and sure enough, the web community embraced it from day one. The approach has been now copied by all online map technology providers.



There are many ground breaking solutions applied in the design of Google map but I want to focus on one more revolutionary idea of Google team. What started as simply clickable markers on the map has now been extended to include polylines and polygons – that is, they have found a way to include vector data utilising VML and SVG capabilities of standard browsers. Sure, there is a capability to generate vector based maps in pure SVG, Flash or Java but no one succeeded to do it on a mass scale in a standard browser environment and without additional plug-ins.

So, where are the limitations then? The main are outlined below, with some commentary.



Limits on Dynamic Content
Pre-designed map tiles approach is brilliant for speed and performance but it does not allow for dynamic map content, unless it is in vector format as points, polylines or polygons (and there is a limitation on how much JavaScript can handle - more on this below) or is delivered using external map server (ie. reverting to traditional approach).

Although Google exposed a number of server side processes to developers, like geocoding, parsing KML, etc, but there is no access to "map server" side of the application simply because Google Map does not have one! The only way to add tiled content without deploying your own map server is to follow Google's tedious process of creating 2-3 different representations of the map for various scales - as graphic images - then cutting them into tiles. There are good tools around to do it but still, it is a very labour intensive approach (for example: Maptiler for images and gmapcreator for GIS data).



Limits on capability to work efficiently with external map servers

Adding content dynamically from external map servers is not without the problems. Even if you master Open Geospatial Consortium’s standards on Web Map Service (WMS) and Styled Layer Description (SLD) there is a good chance that the service you intend to use will not support Mercator projection required for Google maps (eg. very popular topographic map service from demis.nl is one such case). And for obvious reasons Google maps cannot be presented in other popular projections.




Then there is an issue of multiple map tiles again. The bigger the viewing area, the more 256px by 256px tiles are required to present the complete map on the screen. Each tile is a single call to external map server and, with dynamic data, each tile has to be generated from scratch. Each pan and zoom on Google Map translates to multiple calls to external map server. In times of high demand this may flood map server with requests and bring it to halt.



Limited capacity to display vector data

Ability to display vector data in Google map created unreasonable expectations within developer community as to its capacity to handle such data. Problems with displaying tens of thousands of points on the map led to development of add-on libraries to efficiently manage display of such large datasets. Google also implemented line encoding for handling more complex vector line segments. However, unlike Java or Flash/Flex plug-ins, applications reliant purely on Javascript have very limited access to computer processing power. Therefore the ability of Google map to display complex shapes from vector data will always be limited.



Browser memory leaks

There is another limitation often overlooked by developers although this is not Google map issue but rather relating to bad programming practices. Continuous refresh of dynamic content (ie deleting and creating new objects), if not programmed properly, may lead to a gradual build up in memory use by the browser - to the extent that the computer will stop responding to user actions. It is not a big issue with smaller applications used for a few minutes at a time but becomes a significant problem when they get more complex and are used on continuous basis.




So, in conclusion, can Google Map ever aspire to be a fully fledged GIS?

There is definitely a chance it will evolve into a very powerful tool, sufficient for many common GIS applications. However, to be considered in this category it requires at least two functional additions:

1. Map server (ie a web service that would convert user data into an image representation and deliver it back as tiles, same concept as Google chart service);and

2. Dynamic vector data management module – server and client side (ie. server side to serve generalised data suitable for the current map zoom level and client side to efficiently store that data for the session and to display appropriate level of detail without the need to continuously download the data from the original source).



Google has already come up with the initiative to store and distribute geographic data for anyone free of charge so, the two above would be a natural progression in the quest to efficiently serve that data.

Wednesday, June 17, 2009

A Little Tidbit

I'm gearing up for a new series of posts based on some fascinating reading I've been doing lately. I'm not going to spill the beans, but I will give you a little hint, from a paper written by Dr. Robert S. Corruccini, professor of anthropology at Southern Illinois university. I just came across this quote and it blew me away. It's so full of wisdom I can't even believe I just read it. The term "occlusion" refers to the way the upper and lower teeth come together, as in overbite or underbite.
Similar to heart disease and diabetes which are "diseases of civilization" or "Western diseases" (Trowell and Burkitt, 1981) that have attained high prevalence in urban society because of environmental factors rather than "genetic deterioration," an epidemiological transition (Omran, 1971) in occlusal health accompanies urbanization.

Western society has completely crossed this transition and now exists in a state of industrially buffered environmental homogeneity. The relatively constant environment both raises genetic variance estimates (since environmental variance is lessened) and renders epidemiological surveys largely meaningless because etiological factors are largely uniform. Nevertheless most occlusal epidemiology and heritability surveys are conducted in this population rather than in developing countries currently traversing the epidemiological transition.
In other words, the reason observational studies in affluent nations haven't been able to get to the bottom of dental/orthodontic problems and chronic disease is that everyone in their study population is doing the same thing! There isn't enough variability in the diets and lifestyles of modern populations to be able to determine what's causing the problem. So we study the genetics of problems that are not genetic in origin, and overestimate genetic contributions because we're studying populations whose diet and lifestyle are homogeneous. It's a wild goose chase.

That's why you have to study modernizing populations that are transitioning from good to poor health, which is exactly what Dr. Weston Price and many others have done. Only then can you see the true, non-genetic, nature of the problem.

Saturday, June 13, 2009

Law of “exponentialism” of online advertising revenue and online business value

Ok, there is no such term like "exponentialism" but English language is so versatile that allows “creative extensions”. This one meant to highlight the fact that revenue from online advertising and value of websites tend to increase “exponentially” in relation to increase in traffic. To illustrate the point, let’s consider a few examples.

Facebook has reportedly 100 million monthly users and attracts revenue of $350+ million per annum. The ratio of annual revenue to monthly traffic is 3.5. Or, in other words, total annual advertising revenue equates to $3.50 per each monthly visitor.

John Chow blog has 200,000 “uniques” a month and the author publicly admits to approx $500,000 a year in revenue (not all advertising but close) giving it a multiple of 2.5.

Small sites that run predominantly Adsense advertising can count on revenue multiple of just 0.1 to 0.2 (ie. roughly, equivalent of $1,200 a year per 10,000 monthly users).

In addition, as highlighted in a recent post “Millionaires abound! Business lesson from Facebook…” large sites are able to attract investors at much higher value per visitor ratio than smaller sites. In case of Facebook, the recent investment by Russia’s Digital Sky Technologies, values each visitor to that site, in any given month, at $100 per head. Such valuation is envy of all aspiring Internet entrepreneurs.


Therefore, in the light of the above, it can be concluded that:

First law:
“Advertising revenue per visitor increases exponentially with the increase in monthly traffic.”

[updated 25/06/2009]


Second law:
“The value of the Internet site increases exponentially with the increase in monthly traffic.”



What does it all mean? Well, if your motivation is to make some money from advertising on your website or blog, or from selling it at some point, the first and most important objective is to get as highest traffic as possible. Profitability will come later… Many successful Internet start-ups seem to follow this strategy.

I estimate that the “sweet spot” is around 100,000 visitors per month (ie. when you can start considering giving up on your day job!) but it will depend on the market and the niche you choose, and if you can do a good job finding advertisers!

End Note:

To prove the validity of the above laws "beyond a reasonable doubt" it would be good to collect more substantial evidence. Please help me in gathering that evidence! If you have a site or find some relevant information on the web just reply to this post with a name of the website, monthly traffic (unique visitors), advertising revenue (annual) OR sale price/ investment value, and the source (if this is your own site please state, otherwise provide URL to original information). I will update tables and graphs periodically.

What’s in this for you? Well, the information will be useful as a benchmark to assess whether you are efficiently monetising your assets through advertising or adequately capitalising on asset value while selling all or part of your website. So, please contribute!

Thursday, June 11, 2009

Change of blog address. Please visit goinglocaltravel.com

Hello. I'm experimenting in Wordpress. You can find my new blog here - www.goinglocaltravel.com. Please bookmark this address. This is where I am adding my new posts.

Saturday, June 6, 2009

Printing aus-emaps.com maps

Option 1: Using browser “Print” function to print a page

This option will work well for maps that fill the entire page (like in Hazards Monitor series or Postcode Finder). However, it is not optimal for printing the front page map (for which it is best to use “print” function described in Option 2 below).

Steps:
  1. From your browser menu select: File/Print Preview/ to view how the page will look.
  2. While in “Print Preview” mode, select Page Setup to set printing options (eg. turn on/off background images, show/remove header and footer and adjust information to be printed, like date, title page, etc – options will vary from browser to browser).



Hint:
  • See section on browser limitations (below) to troubleshoot printing anomalies.

Option 2: Using aus-emaps.com “print” function

This option is recommended when you want to print just a map and displayed information, without other marginalia that appear on the front page.

Steps:
  1. Print icon is located just above the map. Click on it to generate “printer friendly page”.
  2. Adjust zoom level and map extents, as required.
  3. Use browser print function to print the map (described in Option 1 above)

Advanced Use:
  1. Locate place of interest using search function (eg. open “Searches” tab under the map and type in address into a search box).
  2. Details about the location appear below the search box. Click “save” to add information about that place to a list of “saved locations”.
  3. Repeat 1 and 2 above to add more locations, as required.
  4. Select “printer friendly page” to generate a map depicting locations and showing details of all saved points of interest.
  5. Print map using browser’s print function (as described in Option 1 above)


Hint:

This print option is really handy if you need a single map with multiple locations of interest marked on it. Examples of use:
  • marking location of open houses to visit on the weekend
  • showing locations of weekend garage sales
  • marking locations of your hotel and meeting places for a business trip
  • listing drop-off points and delivery sequence for your track drivers
  • and more…



Limitations how browsers handle map overlays

Please note that not all map overlays can be printed. Eg street view overlay will not print (blue outline nor images), Wikipedia icons either…

Some older browsers, notably Internet Explorer 6, are not able to handle marker images in png format. These will be printed with black rather than transparent background.

Postcode and suburb boundaries can only be printed in IE browser but postcode numbers and suburb names will not print at all (eg. Postcode Finder map).


Use Restrictions

In general, there are no restrictions on printing aus-emaps.com pages for personal use or limited copies for use within a business organisation. However, there are certain requirements and limitations regarding public display or distribution of map information (whether for a fee or free of charge) that are imposed by Google and its data suppliers. Please consult the following pages to ensure that your intended use is within the guidelines:

Google Maps/Google Earth APIs Terms of Service
Google Maps Legal Notices
Permission Guidelines for Google Maps and Google Earth

Friday, June 5, 2009

Whereis maps on TV - another sign of crisis in free to air media?

Only a few years back TV stations were very strict in executing their policy that no third party logos/ references to be shown on graphic materials displayed during transmitted programs. Weather information was the most prominent example of this “policy”. Until very recent there was not even an acknowledgement that it all comes from the Australian Bureau of Meteorology. Not to mention that there were no references to providers of interactive displays of the information. But it is all starting to change now.

Channel 7 is not shy anymore in flashing big Whereis logo and running taglines “get free maps and directions…” during morning weather presentations. Channel 10 has gone even further and now has a weather segment supplied by The Weather Channel – the whole package, including prominent logo display and a presenter! In case of Whereis it could be just paid product placement but it would be a scoop for Telstra if they managed to negotiate “free use of maps for promo” deal. Channel 10 most likely opted for cost cutting and outsourced the whole weather information segment to a third party.

Wednesday, June 3, 2009

Google map v3 enables free mobile browsing for everyone!

Last week Google announced release of version 3 of their map API service. There is no difference how the new map looks on the outside but everything under the hood have been totally redeveloped according to Model–View–Controller (MVC) architectural pattern used in software engineering. As one Google engineer put it, “the primary motivation behind this new version was speed, especially for rendering maps on mobile browsers.”

My initial impression is that improvements are not readily noticeable in a standard desktop environment. But I take Google’s word for it that it will be “fast and furious”. I am starting to hit barriers how far I can extend current applications. For example, all front page map functionality of aus-emaps.com, like searching or viewing extra layers, is loaded only “on demand” to limit the size of initial download and the load on computer memory and processor. The new, high performance, approach to Google map architecture is most welcome.

Google already had a version of maps for mobile phones but it was not available for developers. Version 2 of map API, although could work on mobile browsers, is too big for any advanced applications. The only way for developers to deploy Google maps for mobile browsers was to use scripted static maps (eg. like dial-up version of aus-emaps.com) or develop them only for iPhones. Now everyone can develop and deploy free applications that will work on almost any mobile browser. Here is an example of one of the first deployments of v3 for mobiles from Lonely Planet: