From Pyramids to Meshes – 2006 to 2016

Recently I was looking back at the origins of our work here at Mojolab and I came across an old pair of documents from 2006. At the time I was a fresh engineering graduate working two jobs and trying to get a fellowship at Stanford to work on building what I called at the time, an “Alternative Information Architecture” for Chhattisgarh, which was a focus area for two generations of my family. At the time, this is the definition we had of an “alternative network” –

Introduction to Alternative Networks
For any community aiming to share information efficiently with a view to participation in
development, an efficient communication network is mandatory. Communication networks are
usually perceived as being a set of hardware to perform the physical transfer of data and a set of
protocols, which govern the transfer. However, in most definitions of communication networks the user’s significance is somewhat suppressed. The individual using the network is perceived as an external agency, not really a part of the network. The developments that may occur as a result of human interaction with the system are also seldom taken into consideration at the time of design. The result is that the benefits from incorporating each user of the network as a network
component are overlooked. Alternative networks, which aim to diverge from the existing traditional network models, can be of many types. The common factor among all such networks is that they aim to de-centralize and make informal the process of communication.

What’s interesting to note is that while I was able to articulate that the type of network needed would be an “alternative”, I really didn’t have a clear articulation of why it was an alternative or what it was an alternative to. If I was to rephrase myself today, I would call it a hybrid mesh network consisting of multiple channels at multiple layers/levels of the network each connecting to one or many nodes as an alternative to a network of stars or a tree with clearly laid out hierarchies.

The means of change from a hierarchical model to a networked peer driven one would be superseding rather than breaking the hierarchies already present by creating new interlinks complementary to the existing ones.

The notion of incorporating the user would remain as is, though I would further elaborate that the role of the user would be to simply increase his/her own data acquisition and dissemination capability by learning more skills. The measures of this could be the number of channels that a user is adept at using for communication.

At the time, the essential or defining characteristics that I could come up with for any such alternative network were as follows:

Definition of an Alternate Information Architecture
The most attractive fact about an alternative architecture is that it is extremely conducive to
customization.
The primary concepts that this alternative architecture will use are
• Layering
• Self Organization
• Self Sustenance
• Robustness and Self Healing
• Collective Intelligence

In hindsight, the layering model I had in mind at the time was somewhat limited by my limited field experience. As a result, the descriptions of the layers below end up taking a rather hierarchical form, which is counterproductive to the goal of decentralization.

Tier 1, Information Gathering
Grassroots level information gathering, by “barefoot journalists”, who would be trained minimally.
The data at this stage could be in audio form, or, if the “barefoot journalist” is educated to some
basic level, then written formats could also be used.
The skills required by the members of this tier would typically include: a) an ability to recognize
information relevant to the community, b) self-motivation and enthusiasm and c) basic aptitude
towards learning.
The members at his tier could also be involved in making efficient use of the oral tradition of the
region to disseminate information.
Tier 2, Compilation and basic editing
Compilation of information gathered by “barefoot journalists” into digital form and basic editing.
This would involve conversion of audio material into text for dissemination on the Internet, basic
editing on the text matter to ensure that relevance and consistency is maintained and finally
categorizing the information. The skills required by members at this level would be a) ability to
use a computer with reasonable proficiency b) ability to analyze large amounts of data c) basic
aptitude for editing d) ability to recognize relevant information
Since this tier is intended to be involved in basic editing and compilation, a fundamental level of
error checking would be required. Also on the information dissemination side, this tier would be
responsible for some routing of the information to the various sections of the community.
The members at this tier would also be responsible for spreading awareness about the usage of
the architecture to obtain information. They would be required to demonstrate the website use to
other members of the community. In addition to this, they could be involved in the dissemination
of audio-visual content in areas where literacy and connectivity is low.
Tier 3, Planning, Management and Dissemination
Collection of compiled data from Tier 2, classification of the data into different sections for easy
user access and dissemination through the CGNet website. This level would involve some degree
of technical know-how. The skills required for this tier would include

  1. ability to create and manage web content,
  2. basics of local language computing and
  3. ability to present information in a manner that can be effectively received and used by the community.

This tiered architecture was in fact implemented in the form of CGNet Swara by 2010, where the Tier 1 participants reported and listened to content using mobile phones, which require a minimal amount of training to use effectively. Tier 2 consisted of a verification and moderation network and Tier 3, i.e. the online tier was social media.

This implementation was key in helping us understand the inherent limitations of the hierarchical layering structure. As the diagram below shows, the flow of information in a hierarchical structure is vertical. pyramidWhile this does not prevent horizontal information flows, it also does not acknowledge or leverage them. As a result, the costs of information transfer in a hierarchical system rise as the system scales since all information must first rise from the bottom to the top and then descend again, ostensibly in a filtered and value added form. However, the cost benefits of such value addition must be compared to the cost benefits of horizontal information transfer where members are free to share information irrespective of their position in the hierarchy and the “quality” and “credibility” of a source is based on peer review rather than hierarchical accreditation.

The other issue with the model as described in the paper is that it assumes a uniform access to the Internet. Considering that I wrote this in 2006 and Internet access still remains very fragmented even 10 years later, clearly shows the fallacy of this assumption. Further more, while the model takes into account the effort in curating incoming information from the grassroots, it does not acknowledge or measure the effort in sending information to the grassroots.

Given the condition of Internet penetration and the asymmetry of useful information vs noise on the Internet, the data dissemination cycle needs at least as much effort as the data acquisition cycle.

When one takes this into account, it becomes evident that any entity that implements equal rigor in data collection and dissemination can only be a peer with the other elements of the network and the efficacy of an entity as a network member would be measured not by hierarchical certification but by the number of channels it operates in. In other words, PiFiTV-FullMesh2the credibility of an information source would be based on number of people it interacts with and number of people who interact with it, rather than by how effectively it talked about its “processes” and how many people it “certified”.

The result will likely be much messier…but much more colorful!

The full paper and abstract are linked below. I will be posting further updates to this train of thought as the analysis continues.

 

CGNet RDVP Abstract

CGNetIA Project Proposal

Disks on A Plane – Building The New Information Supply Chain

FlyingDiskBy way of open source development where possible and piracy where not, new innovation is enabling communities and individuals to experiment with building, owning and operating communications infrastructure. This is enabling strengthening of localized communication. At the same time the global nature of the Internet and the new opportunities it presents for peer to peer collaboration are resulting in virtual communities being created on the Internet that augment local communities and provide access to content that would not be available locally. New hybrid models of communication and information sharing are being invented and experimented with. In addition to Mojolab’s own experiments with mechanisms such as data muling and locally operated COWMesh networks, Learning Equalities’ Kolibri project seeks to bring the benefits of a digital classroom and current content to communities beyond the pale of the Internet.

We re-did our thought experiment of muling data on a railway journey, this time with a larger data packet size and then we tested it out in part by muling some data back and forth between Bangalore and Kakrana, a remote village in Madhya Pradesh, via Bhopal.

The though experiment revealed the following –

Screenshot from 2016-02-24 13:56:05Screenshot from 2016-02-24 10:02:28

 

 

 

 

 

 

 

On practical experimentation –

I downloaded a total of 25GB of data on my home connection –

Initial data download (MB) 25600
Rate of data acquisition (INR/MB) 0.0111816406
Cost of data acquisition (INR) 286.25

 

Total data transacted over journey

 

Date Data Rcvd (MB) Data Given (MB) What Cost
10/04/16 42.6 0 R resources 0
14/04/16 272998.4 0 Ranikajal Data 0
14/04/16 27238.4 Arjun Brought 0
Total (MB) 300279.4

 

Had I attempted to perform the same transaction over a 3G connection @ INR 250 per GB or ~INR 0.25 per MB, the cost of transaction would have been .25 * 300279.4 = INR 73310 plus change.

As it happens, the cost of my travel (by air conditioned train) cost less than INR 5000 in total (up and down). This puts the cost per MB of my mule based data transaction at about INR .015 per MB.  Which is about 1/14th of the cost per MB that 3G would have provided.

You can imagine our enthusiasm therefore to get tools like Kolibri and KALite on to a plane quickly!

 

Barriers to Scale – Growth vs Replication

growth vs repThis time in 2014 I was working the Ashoka Globalizer team in preparation for the Globalizer Summit in Bonn in June of that year to evaluate the work we were doing in terms of its readiness for global scale. The Globalizer exercise came with the great privilege of working with a world class consulting team comprising of one global entrepreneur and Ashoka fellow and two analytical experts from McKinsey and Co. Working with this distinguished team, we crunched up the numbers from our work with CGNet Swara, a pioneering citizen journalism effort and our flagship project at the time. Our goal had been to analyze whether the model, i.e. using a localized interactive voice response system to create a voice based bulletin board or “audio portal” for newsgathering and community interaction from remote areas where media and the Internet had not penetrated, was ready for global application and scale.

The indicators at the time seemed to suggest that it was, since we had replicated the technology successfully for use-cases in Afghanistan and Indonesia. However, on analyzing the numbers we realized that while the model was very successful in terms of bringing content out of and into media/connectivity dark regions, the cost at which it did so was not strictly sustainable unless an external party or parties were willing to support it.

While differential costs of communication and data access as well as policy plays a major role in determining the cost of a technological intervention of this nature, an equally significant role is played by the design of the system and its intended scaling strategy.

For a system that seeks to retain its character through the course of scale, whether in the interest of maintaining quality or preserving brand value, the costs of scaling are significantly higher.

In the case of CGNet Swara, what we learned was in line with what we had been told by experts from Google fairly early on. The two major impediments to scale were 1) the fact that the project sought to remain completely free for access by users, on the pretext that they would otherwise find it too expensive to use and 2) the need to maintain a centralized moderation strategy that filters content prior to release.

Both of these strategies were initially employed to engage users and maintain quality.

To attract a largely low income population to use the system, the design was set up to respond to a “missed call” placed by end users with an outgoing call from the system. The end user therefore ended up paying nothing. Had a telecom operator been supporting the system, this would have been infinitely sustainable. However, the project was paying retail prices to purchase telephone airtime, which while infinitely cheaper than broadcast media was still quite expensive at scale, resulting in a large cashflow requirement. Had this been distributed by encouraging more replications, the costs of data collection could have been distributed.

Since the platform was initially implemented by individuals who were solely liable for the content being released, the choice to filter certain content was made to ensure continued operation of the platform free from state intervention. What the team failed to take into account was that this was relevant only as far as the platform was publishing news anonymously on behalf of callers. When the decision was made to publish the users name in the interest of transparency, the responsibility of the content was already on the user. At this point, the filtering mechanism could have changed from a “screen and release filtered content” to “publish and remove reportedly offensive content”. The requirement would have been to explain the liability of publication clearly to users via a terms of use communication.

Had these two mechanisms been employed, the system could have evolved to being an aggregator of content from several replications, which would have been a far more sustainable and scalable option.

We have seen similar patterns in most other technological interventions. The need to maintain centralized control and ownership often prevents organic scale that could come from free replication.

With this learning, we have since been working on ways to enable more replication than growth in our projects. The growth element is now redirected towards growing capacity as well as enabling growth in numbers of replications rather than growing a single system

For example, with our COWMesh project , we have taken the approach of not directly implementing solutions for communities but to enable communities to set up their own systems after observing what is possible on demonstrative systems which Mojolab maintains and travels to community locations with.

Given the gap in skill availability at the grassroots and skill requirement to use open source tools, we have simultaneously been working on setting up mechanisms to train more and more people at the grassroots who can then utilize their skills locally. This element of our work is where we currently need the most support.

We therefore invite collaboration from anyone who wish to improve skill availability at the grassroots.

 

Red Queen! – Low Tech Dev in a High Speed World

redqueenI’m currently fighting a battle with a hardware vendor to get a piece of equipment replaced. The particular equipment in consideration is a router to be used by a school in remote rural Madhya Pradesh that has been functioning for over a decade with very little resources. Should my efforts be successful, they get email access. However, at the moment, the device in question is refusing to cooperate with the design I am trying to implement with it, resulting in my needing to get a replacement or a resolution from vendor or manufacturer. Should I fail to get such replacement or resolution, I will be left with a choice between absorbing the loss of a router (i.e. INR 7000) or passing it on to the “customer” i.e. the school.

In a regular commercial scenario, this would be a no brainer. The dealer/vendor NEVER absorbs a loss. However, commercial scenarios often assume peer relationships where none exist. For a   corporation or research lab with a large technical budget the loss of one piece of equipment would not be a huge one. Collateral damages of research are considered an acceptable expense head in most cases. For low budget exercises on the other hand, each setback represents real expense and loss of opportunity to use the same resources elsewhere. As a result either I as the researcher or the school as the consumer has to take a hit.

Our current situation points to an increasingly prevalent situation faced by people who implement technology for low income/low commercial consumption groups. Open source technology, while cheap and free to use is seldom stable enough to be used by consumers with limited skill. The situation is not made easier by hardware manufacturers who sometimes by design and sometimes by accident develop hardware which does not meet existing standards, resulting in new development being needed before the hardware can be used with open source systems. Development, irrespective of who does it, comes at a price measurable in person-hours, which adds to the cost that has to ultimately be borne by the end user. Finally, governments have been increasing the level of control that they exercise on technology steadily, meaning that several open alternatives are now simply unavailable by statute.

As a result, groups like the Mojolab, who are at the border of technology development and usage are faced with the need to constantly evolve strategies and often use non intuitive means to achieve intuitive ends. It is akin to running very fast simply in order to stay in the same place. In evolutionary terms, this is called the Red Queen phenomenon, inspired by the Red Queen’s chessboard sequence in Alice in Wonderland

“Well, in our country,” said Alice, still panting a little, “you’d generally get to somewhere else—if you run very fast for a long time, as we’ve been doing.”

“A slow sort of country!” said the Queen. “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”

Lewis Carrol, Through The Looking Glass, Chapter 2

 

 

COWMesh Ranikajal #0 – Building Networks in Connectivity Choked Rural India

DSC03027Earlier this month I was at the Ranikajal Jeevanshala school in Kakrana, Madhya Pradesh, studying the region for possible applications of the COWMesh network design that we have been working with.

Connectivity in India is interestingly distributed. The Internet is penetrating every day into new and hitherto unreached areas. However, and this is a big however, the quality of connectivity varies drastically between different areas. In most newly connected locations the best possible data connection that can be found is a 2G network that experientially feels like an old 56k modem line to use.

Even this connectivity is quite welcome as it is perfectly adequate for text browsing and email. DSC03056However, GMail and other popular webmail services now come with extremely heavy interfaces (all that AJAX and Javascript-y stuff that keeps pulling data from the server “on-demand”), which are extremely hard to use on slow connections. Furthermore, even where a fast connection (like a 3G) is available the cost of data is quite high, considering that rural India is has traditionally been viewed as a low income region. The current retail rate for 3G is about INR 250/GB. This means that a standard 120 minute mp4 video would cost almost 180-200 rupees. Compare this with the price of  DVDs which often contain several 120 minute videos for the measly price of INR 40, or even with the price of an SD card, which retails at about INR 280 for a 16 GB card, which can be re-used and overwritten.

DSC03036The Ranikajal team, which has recently seen the addition of outstanding ex-DAE scientist Shri Swapan Bhattacharya, originally envisioned using the COWMesh to create a wireless link to the nearest location where 3G connectivity is available to bring fast Internet to the school.

As part of our survey and following discussion we observed that the vantage points where we would need to set up hops in order to connect Kakrana to the nearest 3G connected locations, i.e. Dahi or Kulvat, were largely uninhabited. Therefore setting up and maintaining routers in those locations would not be feasible.

Further, 2G network access was working via Airtel on the school campus, at a hilltop and it appeared to be far more feasible to set up a local mesh to connect the top of the hill to the bottom and then provide a link into the village. Since full scale browsing on the new graphical web is not really feasible using the 2G network, we concluded that we need to set up a mechanism to provide email access. The way this would work would be that a local mail server running on a Raspberry Pi would be attached to the wireless router on the hill. The Pi will be connected to a smartphone over a usb tether to provide an internet connection, which will be share over the mesh. A dongle may also be used, though our experience tells us that mobile phones are best optimised for using low bandwidth high latency connections, provided of course that advertisement downloading apps can be weeded out.

DSC03100The mail server will download emails from the users accounts whenever the connection permits. Users in the school and anyone else on the mesh with an account on the local mail server can access and send emails through it. This would allow users to use email without the slow experience of trying to load a rich web interface through a slow connection.

System is currently being built. More posts to follow on testing and application.

(Update – We are using OfflineIMAP and Postfix with a Roundcube interface at the moment)