There’s been a lot of chatter in the Indian data science and data enthusiast community about how the 2014 Indian elections have marked a watershed moment in the way journalists and analysts use data to tell stories, as well as how campaigns target voters.

Here’s a talk (via @Datameet, via Hasgeek) on the ways in which tech has influenced campaigns.

DataMeet is soliciting entries for an upcoming conference in July that aims to review how data transformed Elex 2014. Take a look at some of the proposed presentations here, and vote for one. Or suggest your own.

Here’s Anand S of Gramener on how the company created their election visualizations.

The guys at Gramener spoke at Open Data Camp Bangalore, an event focused on data in elections. The sessions look fascinating, but unfortunately, none of the other presentations appear to be online. Check out the full agenda here.

As part of another upcoming H/H event, I got an email from Pykih, a data services startup that worked with FirstPost on the elections. They have a fantastic walk-through on their blog that details how they built out the web app they created for FP. They focus on the fact that they wanted to find an intuitive way to present both national election results and the more fractured regional realities upon which these results depend.

A lot of the credit for this increased use of data goes to India’s growing open data and open knowledge communities. As someone who’s followed along on the listservs and conversations (and co-organized an event in partnership with Datameet before) I’ve witnessed firsthand the dogged determination required to extract, compile and free data that the Indian government either refuses to make available, or makes available in fragmented and effectively useless form. A post by Rukmini Srinivasan on the Hindu’s Data Delve blog offers insight into how essential and transformative the Open Data community’s efforts have been. Srinivas Ramani’s constituency-wise map of voter turnout (which provides figures for both 2009 and 2014) in India is the type of project that could transform voter awareness in India if it were applied to other data sets. Imagine projects like Ramani’s about civic services, or constituency-wise health concerns? Imagine combining a data-driven analysis like this with the ground-level audio reports gathered by Gram Vaani, CGNet, and others, in an attempt to quantify last-mile effectiveness of government programs?

Elections 2014 created an opportunity for Indian journalists to experiment with the ways in which they use data. They also offered a rare opportunity for companies to sell visualizations to news organizations that might otherwise be wary of large outlays on data projects. Hopefully, elections also made a case for future large data-driven projects, in areas that go far beyond elections.

Well, Brian Stelter certainly hasn’t lost anything in the move from the NYT. Two interviews about highly-publicized and high-profile journalists who are leaving big traditional newsrooms (where they have very good roles) to start something new.

Bill Keller – of NYT – is starting a digital media org that’ll focus on issues and abuses in the American criminal justice system. Laud the motive, appreciate the possibility. Read Stelter’s interview with him.

Ezra Klein’s new Project X – an as-yet-largely-undefined explanatory news site under the Vox Media banner. Here’s Stelter’s interview with Vox CEO Jim Bankoff.

I remember when Andrew Sullivan took the Dish independent, and shock waves went through the industry. Many people ran his numbers and said he’d never get enough subscribers to offset his costs. In the case of Klein, much has been made over Wonkblog’s 4 million visitorship figure (which doesn’t seem all that high).

But it also makes me wonder at this broader trend of top journalists leading major news orgs to start their own digital media ventures. In a piece in Nieman Lab, @kendoctor posited one possible cause: “No printing presses or broadcast pipes.” He says “relatively cheap entry” at about $25 million is now achievable. To be fair, most of these new ventures aren’t projected to be cheap – they’re the opposite. Keller’s site will employ some 20-25 full-time journalists.

Doctor also points to the increasing mobility of individual journalists, who have become content brands in their own right and want to own larger shares of the revenue pie. This isn’t just a greedy move, it’s a smart one, since as he points out, the competition for online ad dollars is already pretty brutal. Doctor says that top journalists have enormous Twitter followings in their own right. To his observation, I’dd add that the digital age has given us very visible and obvious ways to measure the brand value of individual journalists (this brand value probably existed before, but it was lumped in with a newspaper’s general “readership” – which we didn’t parse so closely). Now that we can measure hits, engagement, and RTs for every story and every page, journalists use metrics that even venture capitalists can understand. The shift seems dramatic because it’s taken place so quickly.

These moves make me wonder:

As the competition for talent becomes more intense, are mainstream media organizations losing out?

And if so, what can they do to change that? Walt Mossberg reportedly walked away from AllThingsD in part because he wasn’t offered a sweet enough deal to stay.

In the Millennial age, workers value job “security” less than ever. What do they want? Big challenges, the chance to grow their personal brands, equity or something like it. This is a game where traditional news orgs are going to lose  unless they can find a way to offer that kind of control, risk-taking and profit-sharing.

The other aspect is work culture, and a reputation for valuing really good journalism, rather than just aggregation/virality. A lot of the sites that are coming up plan to spend big (relatively speaking) on talent. Journalists – like all of us – have a tendency to see themselves as indispensable, but the point here is that these ventures are creating jobs for good reporters and writers – for good journalists (something mainstream media companies haven’t been doing for a very long time).

We’ve been talking about “entrepreneurship” within the journalism industry for so long – maybe the chickens are coming home to roost.

It’s natural that those with the ability and talent to take big risks will attract big opportunities. But the industry has also encouraged this new mentality, with conferences, papers and debates over how to bring a culture of entrepreneurship into the journalism industry. Clearly, the efforts are succeeding.

At the end of the day, these projects are great for our industry. But could Keller’s new project – anchored by good reporting and a clear sense of public purpose – have found a home at the NYT itself? And if not, is that something the NYT should be worried about?

(Also see: this story on Vivian Schiller’s move from NBC – and a many-year career in mainstream media organizations – to Twitter.  Another sign of how the competitive landscape for journalistic talent has shifted dramatically in just a few short years. And this isn’t the end, it’s the beginning. As Doctor mentions in the piece above, online ad dollars have gravitated towards big players like Facebook and Google, both of whom are looking for new and creative ways to monetize the relationships between community, technology and content.)

So I was at the final presentations for the MIT Media Lab – WE School design innovation workshop. Why? Firstly, I was scouting for design talent. Secondly, it’s basically a hackathon on steroids, and everyone knows how I feel about hackathons.

Out of thousands of applications, some 300 students/young professionals were selected for the 7-day workshop, and organized into groups working on projects in one of ten tracks. These tracks included banking, grassroots innovation, the future of imaging, etc. Their brief was to visit an institution (bank, school, etc) and then spend the seven days – under the guidance of MIT Media Lab students from the US – creating a working prototype of a futuristic product that solved a problem in their category.

The projects ranged from the fantastical – a group projected images onto fog, another created lightbulbs that jumped when they sensed a person passing under them – to the practical – an e-learning game for girls who drop out of school, a 360-degree camera embedded in a bracelet.

I didn’t see all of these projects, but here’s a video of one of the coolest groups I did meet. They’d created a project that could sense the thickness and size of any surface placed underneath it. The projector would scale the projection to the size of the screen (in the video below, the demonstrator is unfolding a piece of paper. As he unfolds it, the image expands).

They’d also created depth sensation, so that as someone moved the paper up, the image zoomed in or out. Especially fascinated was the way the sensor responded to the picture of a boy. As the paper was moved closer to the sensor, the projector began to cycle through a slide show of the boy aging.

Who wrote the code, I asked. Someone pointed to a skinny kid standing next to his computer. “I’m fourteen,” he said. Geez. (He was a precocious exception. Most of the participants were 18-26 years old). He walked me through the 10,000-line program that controlled the sensors. He had written several sets of code to control the device.

Here’s a video of the team that projected images onto fog. They were part of a track called “Magical interfaces.” Which might explain why, the moment I saw this presentation, I thought of Galadriel’s enormous bowl of magical water from Lord of the Rings. (And now you know why I hang around at hackathons)

Here’s one of a team that connected a laser pointer to a computer mouse. (Ignore my chirpy commentary, please).

Alas, I have no video of this, but one of the teams also designed fire-breathing shoes. (Except that the fire was a specially constructed and contained chemical reaction – or so it looked like.)

Of course, the really fascinating stuff is happening at MIT’s Media Lab, for which this event seems to serve partly as a recruiting exercise. Several professors – including the spectacular Pattie Maes (remotely) – presented the work that their research groups are looking at. Maes spoke about the smarter devices work her group is doing, as well as a dual-sensor system embedded in a ring (the ring reads aloud to blind people.) More of their work on their site.

Ramesh Raskar spoke briefly at the beginning about his group.

The response environments group head, Joseph Paradiso, presented a whole lot of cool stuff happening with sensors. He talked about circuit stickers and this absolutely, insanely cute social story-gathering robot that one of his team mates built. (Seriously, this robot is the cutest thing since Nemo.) I especially liked the idea of the story-gathering robot because of its obvious interfaces with journalistic practice. The idea that we could one day use robots to tell stories – or even do portions of field reporting – is the sort of thing that probably strikes people as sacrilegious now but could one day be a valuable part of SOP.

*The image up top comes from an art project one group created as part of their fabrication prez. The pieces – connected by elastic thread – are supposed to resemble origami.

This report in ET the other day caught my eye. Key stat: “out of the 94 million monthly active [Facebook] users in India, 72 million are men – which amount to a staggering 76% of the total Facebook population in India.” This fact has been widely reported, but I’ve yet to trace it back to an official source (FB’s Q4 calls, slides and statements make no mention).

Now, let’s examine this in closer detail, because I was surprised by these figures. I would have expected a majority of FB’s Indian users to be men (because majority of Indian Internet users are men), but I wouldn’t have expected such a pronounced majority. HOWEVER.

According to the report, FB has 94 million users in India. According to a 6-month-old report in NextBigWhat, FB has 62 million mobile users in India. According to this Tech2 report from September 2013, nearly 90% of India’s mobile Internet users are men.

Let’s assume the same percentage applies for FB. That means that out of 62 million mobile users, some 55.8 million are men.  There’s your gender gap. (Can’t calculate it more exactly, since we can’t assume that the mobile and desktop users for FB are exclusive groups). But we can conclude out of the 22 million Indian women who are regularly using FB, at most one third are accessing on mobile as well. This is an issue for FB only in that its strongest growth has been in mobile, and it increasingly looks like the future of the FB product will be a mobile one.

So does FB India have a woman problem? No, they have a mobile women users opportunity.  Google recently announced “Helping Women Get Online” – a campaign to get 50 million Indian women online (and, it goes without saying, using Google products.) Their pages features messages like “The Internet made simple” and their official blog says “India’s digital gender gap is larger than those of most developing countries and closing it is crucial.”

It’s only a matter of time before FB follows suit.

Fun additional fact: Globally, FB’s users skew slightly female.

Phew – how long has it been since this blog was updated? Too long. I just got off a Knight-Mozilla OpenNews call (so fantastic, I almost don’t want to tell anyone else about them) and as always, I’ve come away with a wealth of great new reading material and research.

Recently, one of the problems that’s taken on a potentially ominous importance for me is the question of pageviews, and more broadly, meaningful metrics for measuring the value of a particular piece of journalism. The debate over pageviews isn’t new, and I suspect it has some of its roots in the (now largely bygone) haughtiness of journalists who felt that journalism should create the debate, rather than reflect it.

For journalists, the first King of Pageviews was arguably Nick Denton, whose Gawker blogs seemed to represent everything we disliked about where our industry might be heading and how little we might one day be paid. It’s funny that Gawker, which we once feared as the End of History, turned out to be a transitional phase. Then there was HuffPo, and now there’s BuzzFeed. And who knows what might come after that.

Certainly, BuzzFeed’s open pandering isn’t what we’d want every news outlet to look like. A few years ago, I remember chatting with a friend who is now a researcher in the UK. He suggested that news articles derived value from the fact that they could inform. Since then, other media thinkers (including Greg Linch, in a blog post) have suggested that “impact” is the metric we should strive to define. Taking Greg a bit further, Aron Pilhofer (in announcing a Knight-Mozilla fellowship project) said that truly impactful journalism should “change laws and lives.”

But none of this really got at the heart of what good news should do, and that’s probably because (as Greg himself says) the value of a particular piece of journalism is a deeply personal thing.

The online advertising climate of then (and now) has flattened news considerably. I’ve been in online journalism long enough to see these metrics evolve, as Google Analytics has given us ever more things we can measure and track. (Devices! Location! Keywords!) Sites like Chartbeat will tell me things like how far people have scrolled down my page. That’s great, but I’m increasingly unsure why I should care. Does even this incredibly nuanced data parsing give me a real insight into why my story matters? (The oldest, most important question of all)

The question of why my story matters is more relevant than ever before.  I tend to think – as do a lot of people who bet on content startups –  that the frantic scrambling for pageviews that characterized some early news startups will give way to a more mature news environment, one in which specialty sites will have even more opportunity to thrive. Sponsored content and subscription-driven sites, which we’ve started seeing much more of, are the early harbingers of this environment.

Many people who work for small but well-intentioned content sites hold onto the hope that we will at some point be able to define (in a more nuanced way than through pageviews, time on site, or even “engagement”) the value that our products provide to our users and to the wider debate.  When it comes to citizen journalism and civic media, the notion of “changing lives” becomes more important and yet also more fraught. For example, community radio networks like CGNet Swara and Gram Vaani have, by bringing rural voices into the mainstream, put pressure on bureaucracies to pave roads and solve crimes in rural areas.  The women’s newspaper Khabar Lahariya has done the same.  A lot of great, life-and-law-changing journalism happens on a community level, and the process by which citizens create their own media is a huge part of that.

To provide another example, a public interest site that provides a platform for an acid attack survivor to speak about her difficult recovery may have changed her life, as well as the lives of those who come into contact with her story.  But advertisers may not pay for that.  We say that good content rises to the top, but that simply isn’t always the case.  Citizen media faces some of the same challenges in distribution and finding an audience that indie films do. That doesn’t mean that these stories aren’t important, or that they shouldn’t be part of the broader discourse.

For community journalism, “impact” can be smaller in scale, but not in importance. Theoretically, most of us agree with that. But then, how to create an environment where communities that might not have the werewithal to support their own media can still tell their stories and thrive? The question of metrics isn’t just about telling meaningful stories, it’s about finding ways for these smaller sites to thrive and stand (financially, anyway) on their own.

Increasingly, we realize that a lot of pageview traffic isn’t necessarily driven by quality, but rather by legacy brands with strong financial backing and the ability to put money and man hours into creating and promoting their stories. (For a fascinating breakdown of how this process of promotion works, see Brian Abelson’s analysis of stories on the New York Times online. He looks at factors like social promotion, time on homepage, etc for more than 20,000 stories, and finds that accounting for these factors alone – never mind the story’s subject matter – can predict a great deal of pageview performance)  If that’s the case, then users’ pageviews are actually being very carefully harvested by certain clever brands. There’s a great deal of potentially valuable content that  readers may never see.

A lot of great stories are born on citizen platforms, and some of these (if they’re picked up by mainstream media) become major talking points, and change national debate. This fact alone should demonstrate that community media can have extraordinary value, especially in terms of the metrics that Pilhofer mentions above.

Personally, I think there is value in the act of storytelling, and in democratizing that act as much as possible. Civic media may not need to make massive profits, but it needs to stay alive. For years now, since the early difficulties of community radio, no one has really been able to define how to create monetizable value in a space where a lot of important and necessary content thrives. Finding different metrics is one answer to the question of how we can take better advantage of the inclusive potential of the Internet.

I know I promised another post about the ComScore report – which I haven’t finished reading.  In the meantime, I had great fun chiming in with the excellent people at BBC India on a G+ Hangout about how social media is (and isn’t) creating change in Indian society.

The other panelists were a motley crew, including entrepreneurs and academics.  We didn’t really get to a consensus about some the main questions – for eg, could social media drive a revolution? – but here’s what I took away.

Community. Indians often complain that this country lacks civic sense.  Social media user data, unfortunately, support that conclusion.  A Pew research report from December 2012 polled social media users in 21 nations and found that a median of 80.5% of social media users in  four Arab Spring countries – Tunisia,  Egypt, Jordan and Lebanon – discussed community issues online.  The median for the 21-nation sample (which included developed and emerging countries) was 46%.  When it came to politics, a median 65% from these four Arab Spring nations talked about political issues online, while the median was 34%.  Meanwhile, what are Indians talking about online? Turns out, 41% of Indian social media users have discussed community issues online, way less than the 21-nation median.  Forty-five per cent (well above the median 34%) have talked about politics, but a whopping 85% talk about music and movies (well above the median 67%).  So in India, at least, religion is no longer the opiate of the masses – Shah Rukh Khan is.

Of course, there are huge issues in comparing these samples.  The Indian user base for social media is dominated by young men and middle-aged women (this bit courtesy of the ComScore report).  Still, it can’t be denied that the difference in the number of people talking about community issues is important, and bears further consideration.

The conversation around how social media drive change depends on defining “community,” and what that terms means for Indians.  After the Hangout officially ended, a few of us stayed to chat, including Ashish Tandon of Gram Vaani and Sachin Taparia of LocalCircles.  GV builds audio networks in rural areas, LC does the same for urban ones.

I asked Ashish whether he’d noticed a sense of community among GV’s target audience (GV’s sets up IVR-based audio networks in rural Jharkhand).  After all, one of the things that we hear a lot is that rural India has preserved a village-y sense of community wellbeing.  Interestingly, Ashish said that one of GV’s big challenges and lessons was that although people within villages often relied upon and found common cause with each other, their notion of “community” was incredibly limited.  Their communities encompassed one or two villages, at most, and sometimes even fewer.  So the challenge was to get rural indians to start looking at community in a broader sense, and finding ways to unite with people from different geographic and ethnic areas.

Meanwhile, Sachin Taparia is attempting to re-define communities for urban Indians.  He mentioned that LocalCircles had a lot of success getting people in neighborhoods to band together to bring about civic improvements (things like getting drain covers replaced, etc).  Technically, almost every neighborhood has a Residents Welfare Association that handles such tasks; in practice, these RWAs are usually staffed by a few well-meaning octogenarians and disregarded by everyone else.  Delhi is not a community-driven kind of place (in fact, it’s famous for a Darwinian social atmosphere.)  However, people in urban neighborhoods face extensive civic problems that can be positively resolved by working together in groups.  By introducing social and digital gathering places for urban communities, Taparia said LC was able to re-invigorate the lapsed notion of community action, and on several occasions prompted urban residents to create positive change.

There is a broad but important connection between how people approach their communities and how they approach their nation as a whole. Once people feel empowered to fix one minor and local problem, they will cast their sights more widely.  Social media are excellent at curating these daily/mundane concerns, and revolutions are born out of high principles (rarely) as well as mundane things like the high cost of onions (often).  India’s political and media elites are aware of this reality, which is why we have things like the Food Security Bill, as well as today’s top story in our leading economic newspaper: “Stinging onions likely to gatecrash [new RBI governor] Rajan’s Party.”  But the divide remains.  If Indians start posting about onions and drain covers as much as they post about MS Dhoni, maybe social media could start a revolution here too.

Several interesting takeaways from this year’s ComScore “Digital Future in Focus India 2013” report.  For the first part of this reading, I look at the basics.  Slides 1-12.  Follow along by downloading the report here.

Basics:

1. 41% (ie, a plurality) of the world’s online audience is APAC-based, and in absolute numbers, this audience is growing. This fact is significant for consumer-facing media companies for many reasons.

• A) The political establishment in many of these countries does not necessarily have an established history of favoring the free press.  From a legal perspective, social networks and certain related online companies are intermediaries, not publishers.  From a practical perspective, though, their expansion will be deeply affected by media and free speech laws in these nations – a trend that we’ve seen emerge in India and that will only intensify.  America and many nations in Europe enjoy a high level of press freedom and protection of what Americans typically consider First Amendment rights.  Such freedom is not the global norm.
• B) Online behavior can and can’t be generalized across APAC. One interesting trend – the increasing emergence of social networking as a standalone and/or “bridge to the Internet” experience, a reality promoted by the runaway success of programs like Facebook for Every Phone.  In India, much is made of the “fragmented” media market – separated by language, earning power, method of consumption and preferences.  This fragmentation will be visible in the online sphere as well.
• C) The emergence of far more robust “alternative” media, with ensuing demands for greater transparency among established media players.  By lowering the barriers to entry for publishing, the Internet erodes the traditional monopolies on information enjoyed by media houses.  Also, on an anecdotal level, I can say that holding “Big Media” to account is a very popular activity in the Blog-verse.
• D) Emergence of publishers/monetizable online communities.  I remember meeting Premesh of Malaysiakini at an industry event a while back and was struck by the love-hate relationship that his site enjoys with regulation. On the one hand, strict government regulation is the entire reason that Malaysiakini evolved and grew – print newspapers weren’t free to criticize ruling authorities. Malaysiakini, digital-only, quickly developed a reputation for its ability and willingness to criticize regimes. Of course, this also led to government raids and extreme difficulty getting advertisers. Malaysiakini survived as a subscriber-supported site, when all its nearby competition is free.  To an extent, this type of community is what the new Dish is supposed to be about.
• E) An urgent need for greater creativity regarding migrating ad revenues to mobiles.  Increased competition for online ad spends as well as low cost per consumer/click/view will combine. We will be seeing much more micro-media and micro-payment, with related opportunities for profitable disruption of the media delivery model.

2.  Report says India is “at a tipping point for online businesses.”  Guys – whether this is true or not, I can’t say. But I can say that every day brings a new ecommerce site into the mix, many of which seem to be powered by little more than a Shopify front and a Facebook ad campaign.  Clearly, some of these sites are achieving stickiness, but how many? Wouldn’t mind seeing a more detailed drilldown of ecomm startup launches and failures.

Tomorrow: Significance of age and gender breakdown, what the heck is up with car rentals and blogs?