Big Data

You auto-complete me: romancing the bot

sumandro — 2019-12-06T05:00:19Z

This is an excerpt from an essay by Maya Indira Ganesh, written for and published as part of the Bodies of Evidence collection of Deep Dives. The Bodies of Evidence collection, edited by Bishakha Datta and Richa Kaul Padte, is a collaboration between Point of View and the Centre for Internet and Society, undertaken as part of the Big Data for Development Network supported by International Development Research Centre, Canada.

Please read the full essay on Deep Dives: You auto-complete me: romancing the bot

Maya Indira Ganesh: Website and Twitter

I feel like Kismet the Robot.

Kismet is a flappy-eared animatronic head with oversized eyeballs and bushy eyebrows. Connected to cameras and sensors, it exhibits the six primary human emotions identified by psychologist Paul Ekman: happiness, sadness, disgust, surprise, anger, and fear.

Scholar Katherine Hayles says that Kismet was built as an ‘ecological whole’ to respond to both humans and the environment. ‘The community,’ she writes, ‘understood as the robot plus its human interlocutors, is greater than the sum of its parts, because the robot’s design and programming have been created to optimise interactions with humans.’

In other words, Kismet may have ‘social intelligence’.

Kismet’s creator Cynthia Breazal explains this through a telling example. If someone comes too close to it, Kismet retracts its head as if to suggest that its personal space is being violated, or that it is shy. In reality, it is trying to adjust its camera so that it can properly see whatever is in front of it. But it is the human interacting with Kismet who interprets this retraction as the robot requiring its own space by moving back. Breazal says, ‘Human interpretation and response make the robot’s actions more meaningful than they otherwise would be.’

In other words, humans interpret Kismet’s social intelligence as ‘emotional intelligence’...

Kismet was built at the start of a new field called affective computing, which is now branded as ‘emotion AI’. Affective computing is about analysing human facial expressions, gait and stance into a map of emotional states. Here is what Affectiva, one of the companies developing this technology, says about how it works:

‘Humans use a lot of non-verbal cues, such as facial expressions, gesture, body language and tone of voice, to communicate their emotions. Our vision is to develop Emotion AI that can detect emotion just the way humans do. Our technology first identifies a human face in real time or in an image or video. Computer vision algorithms then identify key landmarks on the face…[and] deep learning algorithms analyse pixels in those regions to classify facial expressions. Combinations of these facial expressions are then mapped to emotions.’

But there is also a more sinister aspect to this digitised love-fest. Our faces, voices, and selfies are being used to collect data to train future bots to be more realistic. There is an entire industry of Emotion AI that harvests human emotional data to build technologies that we are supposed to enjoy because they appear more human. But it often comes down to a question of social control, because the same emotional data is used to track, monitor and regulate our own emotions and behaviours...

For more details visit http://editors.cis-india.org/raw/maya-indira-ganesh-you-auto-complete-me-romancing-the-bot

Workshop Report - UIDAI and Welfare Services: Exclusion and Countermeasures

vanya — 2019-03-16T04:34:11Z

This report presents summarised notes from a workshop organised by the Centre for Internet and Society (CIS) on Saturday, August 27, 2016, to discuss, raise awareness of, and devise countermeasures to exclusion due to implementation of UID-based verification for and distribution of welfare services.

Introduction

The Centre for Internet and Society organised a workshop on "UIDAI and Welfare Services: Exclusion and Countermeasures" at the Institution of Agricultural on Technologists on August 27 in Bangalore to discuss, raise awareness of, and devise countermeasures to exclusion due to implementation of UID-based verification for and distribution of welfare services [1]. This was a follow-up to the workshop held in Delhi on “Understanding Aadhaar and its New Challenges” at the Centre for Studies in Science Policy, JNU on May 26th and 27th 2016 [2]. In this report we summarise the key concerns raised and the case studies presented by the participants at the workshop held on August 27, 2016.

Implementation of the UID Project

Question of Consent: The Aadhaar Act [3] states that the consent of the individual must be taken at the time of enrollment and authentication and it must be informed to him/her the purpose for which the data would be used. However, the Act does not provide for an opt-out mechanism and an individual is compelled to give consent to continue with the enrollment process or to complete an authentication.

Lack of Adherence to Court Orders: Despite of several orders by Supreme Court stating that use of Aadhaar cannot be made mandatory for the purpose of availing benefits and services, multiple state governments and departments have made it mandatory for a wide range of purposes like booking railway tickets [4], linking below the poverty line ration cards with Aadhaar [5], school examinations [6], food security, pension and scholarship [7], to name a few.

Misleading Advertisements: A concern was raised that individuals are being mislead in the necessity and purpose for enrollment into the project. For example, people have been asked to enrol by telling them that they might get excluded from the system and cannot get services like passports, banks, NREGA, salaries for government employees, denial of vaccinations, etc. Furthermore, the Supreme Court has ordered Aadhaar not be mandatory, yet people are being told that documentation or record keeping cannot be done without UID number.

Hybrid Governance: The participants pointed out that with the Aadhaar (Targeted delivery of financial and other subsidies, benefits and services) Act, 2016 (hereinafter referred to as Aadhaar Act, 2016 ) being partially enforced, multiple examples of exclusion as reported in the news are demonstrating how the Aadhaar project is creating a case of hybrid governance i.e private corporations playing a significant role in Governance. This can be seen in case of Aadhaar where we see many entities from private sector being involved in its implementation, as well as many software and hardware companies.

Lack of Transparency around Sharing of Biometric Data: The fact how and why the Government is relying on biometrics for welfare schemes is unclear and not known. Also, there is no information on how biometric data that is collected through the project is being used and its ability as an authenticating device. Along with that, there is very little information on companies that have been enlisted to hold and manage data and perform authentication.

Possibility of Surveillance: Multiple petitions and ongoing cases have raised concerns regarding the possibility of surveillance, tracking, profiling, convergence of data, and the opaque involvement of private companies involved in the project.

Denial of Information: In an RTI filed by one of the participant requesting to share the key contract for the project, it was refused on the grounds under section 8(1) (d) of the RTI Act, 2005. However, it was claimed that the provision would not be applicable since the contract was already awarded and any information disclosed to the Parliament should be disclosed to the citizens. The Central Information Commission issued a letter stating that the contractual obligation is over and a copy of the said agreement can be duly shared. However, it was discovered by the said participant that certain pages of the same were missing , which contained confidential information. When this issue went before appeal before the Information Commissioner, the IC gave an order to the IC in Delhi to comply with the previous order. However, it was communicated that limited financial information may be given, but not missing pages. Also, it was revealed that the UIDAI was supposed to share biometric data with NPR (by way of a MoU), but it has refused to give information since the intention was to discontinue NPR and wanted only UIDAI to collect data.

Concerns Arising from the Report of the Comptroller and Auditor General of India (CAG) on Implementation of PAHAL (DBTL) Scheme

A presentation on the CAG compliance audit report of PAHAL on LPG [8] revealed how the society was made to believe that UID will help deal with the issue of duplication and collection as well as use of biometric data will help. The report also revealed that multiple LPG connections have the same Aadhaar number or same bank account number in the consumer database maintained by the OMCs, the bank account number of consumers were also not accurately recorded, scrutiny of the database revealed improper capture of Aadhaar numbers, and there was incorrect seeding of IFSC codes in consumer database. The participants felt that this was an example of how schemes that are being introduced for social welfare do not necessarily benefit the society, and on the contrary, has led to exclusion by design. For example, in the year 2011, by was of the The Liquefied Petroleum Gas (Regulation of Supply and Distribution) Amendment Order, 2011 [9], the Ministry of Petroleum and Natural Gas made the Unique Identification Number (UID) under the Aadhaar project a must for availing LPG refills. This received a lot of public pushback, which led to non-implementation of the order. In October 2012, despite the UIDAI stating that the number was voluntary, a number of services began requiring the provision of an Aadhaar number for accessing benefits. In September 2013, when the first order on Aadhaar was passed by court [10], oil marketing companies and UIDAI approached the Supreme Court to change the same and allow them to make it mandatory, which was refused by the Court. Later in the year 2014, use of Aadhaar for subsidies was made mandatory. The participants further criticised the CAG report for revealing the manner in which linking Aadhaar with welfare schemes has allowed duplication and led to ghost beneficiaries where there is no information about who these people are who are receiving the benefits of the subsidies. For example, in Rajasthan, people are being denied their pension as they are being declared dead due to absence of information from the Aadhaar database.

It was said that the statistics of duplication mentioned in the report show how UIDAI (as it claims to ensure de-duplication of beneficiaries) is not required for this purpose and can be done without Aadhaar as well. Also, due to incorrect seeding of Aadhaar number many are being denied subsidy where there is no information regarding the number of people who have been denied the subsidy because of this. Considering these important facts from the audit report, the discussants concluded how the statistics reflect inflated claims by UIDAI and how the problems which are said to be addressed by using Aadhaar can be dealt without it. In this context, it is important to understand how the data in the aadhaar database maybe wrong and in case of e-governance the citizens suffer. Also, the fact that loss of subsidy-not in cash, but in use of LPG cylinder - only for cooking, is ignored. In addition to that, there is no data or way to check if the cylinder is being used for commercial purposes or not as RTI from oil companies says that no ghost identities have been detected.

UID-linked Welfare Delivery in Rajasthan

One speaker presented findings on people's experiences with UID-linked welfare services in Rajasthan, collected through a 100 days trip organised to speak to people across the state on problems related to welfare governance. This visit revealed that people who need the benefits and access to subsidies most are often excluded from actual services. It was highlighted that the paperless system is proving to be highly dangerous. Some of the cases discussed included that of a disabled labourer, who was asked to get an aadhaar card, but during enrollment asked the person standing next to him to put all his 5 fingers for biometric data collection. Due to this incorrect data, he is devoid of all subsidies since the authentication fails every time he goes to avail it. He stopped receiving his entitlements. Though problems were anticipated, the misery of the people revealed the extent of the problems arising from the project. In another case, an elderly woman living alone, since she could not go for Aadhaar authentication, had not been receiving the ration she is entitled to receive for the past 8 months. When the ration shop was approached to represent her case, the dealers said that they cannot provide her ration since they would require her thumb print for authentication. Later, they found out that on persuading the dealer to provide her with ration since Aadhaar is not mandatory, they found out that in their records they had actually mentioned that she was being given the ration, which was not the case. So the lack of awareness and the fact that people are entitled to receive the benefits irrespective of Aadhaar is something that is being misused by dealers. This shows how this system has become a barrier for the people, where they are also unaware about the grievance redressal mechanism.

Aadhaar and e-KYC

In this session, the use of Aadhaar for e-KYC verification was discussed The UID strategy document describes how the idea is to link UIDAI with money enabled Direct Benefit Transfer (DBT) to the beneficiaries without any reason or justification for the same. It was highlighted by one of the participants how the Reserve Bank of India (RBI) believed that making Aadhaar compulsory for e-KYC and several other banking services was a violation of the Money Laundering Act as well as its own rules and standards, however, later relaxed the rules to link Aadhaar with bank accounts and accepted its for e-KyC with great reluctance as the Department of Revenue thought otherwise. It was mentioned how allowing opening of bank accounts remotely using Aadhaar, without physically being present, was touted as a dangerous idea. However, the restrictions placed by RBI were suddenly done away with and opening bank accounts remotely was enabled via e-KYC.

A speaker emphasised that with emerging FinTech services in India being tied with Aadhaar via India Stack, the following concerns are becoming critical:

With RBI enabling creation of bank accounts remotely, it becomes difficult to to track who did e-KYC and which bank did it and hold the same accountable.
The Aadhaar Act 2016 states that UIDAI will not track the queries made and will only keep a record of Yes/No for authentication. For example, the e-KYC to open a bank account can now be done with the help of an Aadhaar number and biometric authentication. However, this request does not get recorded and at the time of authentication, an individual is simply told whether the request has been matched or not by way of a Yes/No [11]. Though UIDAI will maintain the authentication record, this may act as an obstacle since in case the information from the aadhaar database does not match, the person would not be able to open a bank account and would only receive a yes/no as a response to the request.
Further, there is a concern that the Aadhaar Enabled Payment System being implemented by the National Payment Corporation of India (NCPI) would allow effectively hiding of source and destination of money flow, leading to money laundering and cases of bribery. This possible as NCPI maintains a mapper where each bank account is linked (only the latest one). However, Aadhaar number can be linked with multiple bank accounts of an individual. So when a transaction is made, the mapper records the transaction only from that 1 account. But if another transaction takes place with another bank account, that record is not maintained by the mapper at NCPI since it records only transactions of the latest account seeded in that. This makes money laundering easy as the money moves from aadhaar number to aadhaar number now rather than bank account to bank account.

Endnotes

[1] See: http://cis-india.org/internet-governance/events/uidai-and-welfare-services-exclusion-and-countermeasures-aug-27.

[2] See: http://cis-india.org/internet-governance/blog/report-on-understanding-aadhaar-and-its-new-challenges.

[3] See: https://uidai.gov.in/beta/images/the_aadhaar_act_2016.pdf.

[4] See: http://scroll.in/latest/816343/aadhaar-numbers-may-soon-be-compulsory-to-book-railway-tickets.

[5] See: http://www.thehindu.com/news/national/karnataka/linking-bpl-ration-card-with-aadhaar-made-mandatory/article9094935.ece.

[6] See: http://timesofindia.indiatimes.com/india/After-scam-Bihar-to-link-exams-to-Aadhaar/articleshow/54000108.cms.

[7] See: http://www.dailypioneer.com/state-editions/cs-calls-for-early-steps-to-link-aadhaar-to-ac.html.

[8] See: http://www.cag.gov.in/sites/default/files/audit_report_files/Union_Commercial_Compliance_Full_Report_25_2016_English.pdf.

[9] See: http://petroleum.nic.in/docs/lpg/LPG%20Control%20Order%20GSR%20718%20dated%2026.09.2011.pdf.

[10] See: http://judis.nic.in/temp/494201232392013p.txt.

[11] Section 8(4) of the Aadhaar Act, 2016 states that "The Authority shall respond to an authentication query with a positive, negative or any other appropriate response sharing such identity information excluding any core biometric information."

For more details visit http://editors.cis-india.org/internet-governance/blog/workshop-report-uidai-and-welfare-services-august-27-2016

Workshop on Big Data in India: Benefits, Harms, and Human Rights (Delhi, October 01)

vanya — 2016-09-28T05:53:55Z

CIS welcomes you to participate in the workshop we are organising on Saturday, October 01 at India Habitat Centre, Delhi, to discuss benefits, harms, and human rights implications of big data technologies, and explore potential research questions. A quick RSVP will be much appreciated.

Workshop invitation: Download (PDF)

Workshop agenda: Download (PDF)

In the last few years, there has been an emergence of the discourse of big data viewing it as an instrument not just for ensuring efficient, targeted and personalised services in the private sector, but also for development, social and policy research, and formalising and monetising various sections of the economy. This possibility is premised upon the idea that there is great knowledge that resides in both traditional and new forms of data made possible by our digital selves, and that we may now have the capability to tap into that knowledge for insights across diverse sectors like healthcare, finance, e-governance, education, law enforcement and disaster management, to name but a few. Alongside, various commentators have also pointed to the new problems and risks that big data could create for privacy of individuals through greater profiling, for free speech and economic choice by strengthening monopolistic tendencies, and for socio-economic inequalities by making existing disparities more acute and facilitating algorithmic bias and exclusion.

From a regulatory perspective, big data technologies pose fundamental challenges to the national data regulatory frameworks that have existed since many years. The nature of collection and utilisation of big data, which is often not driven by immediate purpose of the collected data, conflict with the principles of data minimisation and collection limitation that have been integral to data protection laws globally. This compels us to revisit existing theories of data governance. Additionally, use of big data in public decision-making highlights the question of how algorithmic control and governance must be regulated. This raises concerns around taking determining a balanced position that recognises the importance of big data, including for development actions, and ensures unhindered innovation with simultaneous focus on greater transparency and anonymisation to protect individual privacy, and various big data risks faced by population groups. In order to answer these questions, we need to begin with identifying the different harms and benefits of big data that could arise through its use across sectors and disciplines, especially in the context of human rights.

This workshop is designed around an extensive study of current and potential future uses of big data for governance in India that CIS has undertaken over the last year. The study focused on key central government projects and initiatives like the UID project, the Digital India programme, the Smart Cities Challenge, etc.

We will initiate the workshop with a detailed presentation of our findings and key concerns, which will then shape the discussion agenda of the workshop. We look forward to discuss aspects of big data technologies through the entry points of harms, opportunities, and human rights.

The final session of the workshop will focus on identifying key research questions on the topic, and exploring potential alliances of scholars and organisations that can drive such research activities.

We look forward to making this a forum for knowledge exchange for our friends and colleagues attending the discussion and discuss the opportunity to for potential collaboration.

RSVP: Please send an email to Ajoy Kumar at <ajoy@cis-india.org>.

Organisers: Amber Sinha <amber@cis-india.org> and Sumandro Chattapadhyay <sumandro@cis-india.org>.

For more details visit http://editors.cis-india.org/internet-governance/events/big-data-in-india-benefits-harms-and-human-rights-oct-01-2016

Workshop on 'Privacy after Big Data' (Delhi, November 12)

sumandro — 2016-11-12T10:14:52Z

The Centre for Internet and Society (CIS) and the Sarai programme, CSDS, invite you to a workshop on 'Privacy after Big Data: What Changes? What should Change?' on Saturday, November 12. This workshop aims to build a dialogue around some of the key government-led big data initiatives in India and elsewhere that are contributing significant new challenges and concerns to the ongoing debates on the right to privacy. It is an open event. Please register to participate.

Invitation note and agenda: Download (PDF)

Venue and RSVP

Venue: Centre for the Study of Developing Societies 29, Rajpur Road, Civil Lines, Delhi 110054.

Location on Google Maps: https://www.google.com/maps/place/CSDS/@28.677775,77.2162523,17z/.

Registration: Complete this form.

Concept Note

In this age of big data, discussions about privacy are intertwined with the use of technology and the data deluge. Though big data possesses enormous value for driving innovation and contributing to productivity and efficiency, privacy concerns have gained significance in the dialogue around regulated use of data and the means by which individual privacy might be compromised through means such as surveillance, or protected. The tremendous opportunities big data creates in varied sectors ranges from financial technology, governance, education, health, welfare schemes, smart cities to name a few.

With the UID (“Aadhaar”) project re-animating the Right to Privacy debate in India, and the financial technology ecosystem growing rapidly, striking a balance between benefits of big data and privacy concerns is a critical policy question that demands public dialogue and research to inform an evidence based decision.

Also, with the advent of potential big data initiatives like the ambitious Smart Cities Mission under the Digital India Scheme, which would rely on harvesting large data sets and the use of analytics in city subsystems to make public utilities and services efficient, the tasks of ensuring data security on one hand and protecting individual privacy on the other become harder.

As key privacy principles are at loggerheads with big data activities, it is important to consider privacy as an embedded component in the processes, systems and projects, rather than being considered as an afterthought. These examples highlight the current state of discourse around data protection and privacy in India and the shapes they are likely to take in near future.

This workshop aims to build a dialogue around some of the key government-led big data initiatives in India and elsewhere that are contributing significant new challenges and concerns to the ongoing debates on the right to privacy.

Agenda

09:00-09:30 Tea and Coffee

09:30-10:00 Introduction

Mr. Amber Sinha and Mr. Sandeep Mertia
This session will introduce the topic of the workshop in the context of the ongoing works at CIS and Sarai.

10:00-11:00 From Privacy Bill(s) to ‘Habeas Data’

Dr. Usha Ramanathan and Mr. Vipul Kharbanda
This session will present a brief history of the privacy bill(s) in India and end with reflections on ‘habeas data’ as a lens for thinking and actualising privacy after big data.

11:00-11:30 Tea and Coffee

11:30-12:30 Digital ID, Data Protection, and Exclusion

Ms. Amelia Andersdotter and Mr. Srikanth Lakshmanan
This session will discuss national centralised digital ID systems, often operating at a cross-functional scale, and highlight its implications for discussions on data protection, welfare governance, and exclusion from public and private services.

12:30-13:30 Digital Money and Financial Inclusion

Dr. Anupam Saraph and Ms. Astha Kapoor
This session will focus on the rise of digital banking and online payments as core instruments of financial inclusion in India, especially in the context of the Jan Dhan Yojana and UPI, and reflect on the concerns around privacy and financial data.

13:30-14:30 Lunch

14:30-15:30 Big Data and Mass Surveillance

Dr. Anja Kovacs and Mr. Matthew Rice
This session will reflect on the rise of mass communication surveillance across the world, and the evolving challenges of regulating il/legal surveillance by government agencies.

15:30-16:15 Privacy is (a) Right

Mr. Apar Gupta and Ms. Kritika Bhardwaj
This brief session is to share initial ideas and strategies for articulating and actualising a constitutional right to privacy in India.

16:15-16:30 Tea and Coffee

16:30-17:30 Round Table

An open discussion session to conclude the workshop.

Speakers

Mr. Amber Sinha

Amber works on issues surrounding privacy, big data, and cyber security. He is interested in the impact of emerging technologies like artificial intelligence and learning algorithms on existing legal frameworks, and how they need to evolve in response. Amber studied humanities and law at National Law School of India University, Bangalore.

E-mail: amber at cis-india dot org.

Twitter: @ambersinha07.

Ms. Amelia Andersdotter

Amelia Andersdotter has been a Member of the European Parliament. She works on practical implications of data protection laws and consumer information security in Sweden, and digital rights in the Europe in general. Presently she is residing in Bangalore, where she is a visiting scholar with Centre for Internet and Society. She holds a BSc in Mathematics.

URL: https://dataskydd.net.

Twitter: @teirdes.

Dr. Anja Kovacs

Dr. Anja Kovacs directs the Internet Democracy Project in Delhi, India, which works for an Internet that supports free speech, democracy and social justice in India and beyond. Anja’s research and advocacy focuses especially on questions regarding freedom of expression, cybersecurity and the architecture of Internet governance. She has been a member of the of the Investment Committee of the Digital Defenders Partnership and of the Steering Committee of Best Bits, a global network of civil society members. She has also worked as an international consultant on Internet issues, including for the Independent Commission on Multilateralism, the United Nations Development Programme Asia Pacific and the UN Special Rapporteur on Freedom of Expression, Mr. Frank La Rue, as well as having been a Fellow at the Centre for Internet and Society in Bangalore, India.

Internet Democracy Project: https://internetdemocracy.in.

Twitter: @anjakovacs.

Dr. Anupam Saraph

Anupam Saraph has extensively researched India's UID number that has been widely regarded as the game changer in development programs. It has come to be linked with both public and private databases and become the requirement for access to entitlements, benefits, services and rights. Dr. Saraph, who has the design of at least two identification programs to his credit has researched the UID’s functional creep since its inception.

He has been dissecting the myths of what the UID is or is not. He has also tracked the consequences of its linkages on databases that protect national security, sovereignty, democratic status and the entire banking and money system in India. He has also highlighted the implications of its use for targeted delivery of cash subsidies from the Consolidated Fund of India. He has written and lectured widely about the devastating impact of the UID number on development programs, national security and the governability of India.

As a Professor of Systems, Governance and Decision Sciences, Environmental Systems and Business he mentors students and teaches systems, information systems, environmental systems and sustainable development at universities in Europe, Asia and the Americas. He has worked with the Rensselaer Polytechnic Institute, Rijksuniversitiet Groningen, RIVM, University of Edinburgh, Resource Use Institute, Systems Research Institute among others. Dr. Saraph has had the unique distinction of being India’s only person who has held the only office of a City CIO in India, in a PPP arrangement with government, industry and himself. He has also been the first e-governance Advisor to a State government. Dr. Saraph has held CxO and ministerial level positions and serves as an independent director on the boards of Public and Private Sector companies and NGOs. He is also the President of the Nagrik Chetna Manch, an NGO charged with the mission to bring accountability in governance.

Dr. Saraph is also actively engaged in civil society where he participates in several environmental, resource and nature conservation initiatives, has authored draft legislations for river and natural resource conservation, right to good governance and has contributed to governance, election and democratic reforms. Dr. Saraph is a regular columnist in newspapers and writes on issues of governance, future design, technology and education from a systems perspective.

As a future designer and recognized as a global expert on complex systems he helps individuals and organisations understand and design the future of their worlds. Together they address the toughest challenges, accomplish missions and achieve business goals. He also supports building capacity to address the challenges of today as well as to build future designs through teams and effective leadership. Since the eighties Dr. Saraph has modeled complex systems of cities, countries, regions and even the planet. His models have been awarded internationally and even placed in 10-year permanent exhibitions.

Dr Saraph works with business and government executives, civil society leaders, politicians, generals, civil servants, police, trade unionists, community activists, United Nations and ASEAN officials, judges, writers, media, architects, designers, technologists, scientists, entrepreneurs, board members and business leaders of small, mid and large single and trans-national companies, religious leaders and artists across a dozen countries and various industry sectors to help them and their organisations succeed in their missions. He advises the World Economic Forum through its Global Agenda Council for Complex Systems and the Club of Rome, Indian National Association as a founder life member.

Dr Saraph holds a PhD in designing sustainable systems from the faculty of Mathematics and Natural Sciences of the Rijksuniversiteit Groningen, the Netherlands.

Website: http://anupam.saraph.in.

Twitter: @anupamsaraph.

Mr. Apar Gupta

Apar Gupta practices law in Delhi. He is also one of the co-founders of the Internet Freedom Foundation. His work and writing on public interest issues can be accessed at his personal website www.apargupta.com.

Twitter: @aparatbar.

Ms. Astha Kapoor

Astha Kapoor is a public policy strategy consultant working on financial inclusion and digital payments. Currently, she is working with MicroSave. Her tasks involve a focus on government to people (G2P) payments - and her work spans strategy, advisory and evaluation with the DBT Mission, Office of the Chief Economic Advisor, NITI Aayog and ministries pertaining to food, fuel and fertilizer. She recently designed a pilot to digitize uptake of fertilizers in Krishna district, and evaluated the newly introduced coupon system in the Public Distribution System in Bengaluru.

Twitter: @kapoorastha.

Ms. Kritika Bhardwaj

Kritika Bhardwaj works as a Programme Officer at the Centre for Communication Governance (CCG), National Law University, Delhi. Her main areas of research are privacy and data protection. At CCG, she has written about the privacy implications of several contemporary issues such as Aadhaar (India's unique identification project), cloud computing and the right to be forgotten. A lawyer by training, Kritika has a keen interest in information law and human rights law.

Centre for Communication Governance, NLU Delhi: http://ccgdelhi.org.

Twitter: @Kritika12.

Mr. Matthew Rice

Matthew Rice is an Advocacy Officer at Privacy International working across the organisation engaging with international partners and strengthening their capacity on communications surveillance issues. He has previously worked at Privacy International as a consultant building the Surveillance Industry Index, the largest publicly available database on the private surveillance sector ever assembled. Matthew graduated from University of Aberdeen with an LLB (Hons.) and also has an MA in Human Rights from University College London.

Privacy International: https://privacyinternational.org.

Twitter: @mattr3.

Mr. Sandeep Mertia

Sandeep Mertia is a Research Associate at The Sarai Programme, Centre for the Study of Developing Societies, Delhi. He is an ICT engineer by training with research interests in Science & Technology Studies, Software Studies and Anthropology. He is conducting an ethnographic study of emerging modes of data-driven knowledge production in the social sector.

Sarai: http://sarai.net.

Twitter: @SandeepMertia.

Academia: https://daiict.academia.edu/SandeepMertia.

Mr. Srikanth Lakshmanan

Srikanth is a software professional with interests in Internet, follower of Internet policy discussions, volunteers for multiple online campaigns related to Internet. He is also fascinated by FOSS, opendata, localization, Wikipedia, maps, public transit, civic tech and occasionally contributes to them.

Site: http://www.srik.me.

Twitter: @logic.

Mr. Vipul Kharbanda

Vipul Kharbanda is a consultant with the Center for Internet and Society, Bangalore. After finishing his BA.LLB.(Hons.) from National Law School of India University in Bangalore, he worked for India’s largest corporate law firm for two and a half years in their Mumbai office for two years working primarily on the financing of various infrastructure projects such as Power Plants, Roads, Airports, etc. Since quitting his corporate law job, Vipul has been working as the Associate Editor in a legal publishing house which has been publishing legal books and journals for the last 90 years in India. He has also been involved with the Center for Internet and Society as a Consultant working primarily on issues related to privacy and surveillance.

For more details visit http://editors.cis-india.org/internet-governance/events/privacy-after-big-data-delhi-nov-12-2016

Vulnerabilities in the UIDAI Implementation Not Addressed by the Aadhaar Bill, 2016

Pooja Saxena and Amber Sinha — 2016-03-21T08:33:53Z

In this infographic, we document the various issues in the Aadhaar enrolment process implemented by the UIDAI, and highlight the vulnerabilities that the Aadhaar Bill, 2016 does not address. The infographic is based on Vidushi Marda’s article 'Data Flow in the Unique Identification Scheme of India,' and is designed by Pooja Saxena, with inputs from Amber Sinha.

Download the infographic: PDF and PNG.

Credits: The illustration uses the following icons from The Noun Project - Thumpbrint created by Daouna Jeong, Duplicate created by Pham Thi Dieu Linh, Copy created by Mahdi Ehsaei.

License: It is shared under Creative Commons Attribution 4.0 International License.

For more details visit http://editors.cis-india.org/internet-governance/blog/vulnerabilities-in-the-uidai-implementation-not-addressed-by-the-aadhaar-bill-2016

Vidhi Doshi - Fingerprint Payments Prompt Privacy Fears in India (The Guardian)

Vidhi Doshi — 2017-02-13T09:21:42Z

This article by Vidhi Doshi on the use of Aadhaar-based payments by private companies in India was published by The Guardian on February 09, 2017. Sumandro Chattapadhyay is quoted in the article.

Originally published by The Guardian.

For two years, Indian officials have been trawling the country, from city slums to unelectrified villages, zapping eyeballs, scanning fingerprints and taking photographs.

Last month, Indian shoppers started to see the results. With the launch of a government-backed fingerprint payment system, tied to India’s growing biometric data bank, registered citizens can – in theory at least – now pay for things with the touch of a finger.

India’s extraordinary biometric database, named Aadhaar after a Hindi word for ‘foundation’, is the biggest of its kind in the world. It was initially sold to the public as a welfare delivery mechanism that would ensure the country’s 1.25bn citizens were each receiving the right quantity of subsidised rice or cooking fuel, while weeding out fraudsters.

But now this pool of more than a billion people’s biometric data is being used by banks, credit checking firms and other private companies to identify customers, raising questions about privacy and security.

As one of his flagship policies, prime minister Narendra Modi pledged to create a “digital India” in which the country’s cash-centric economy would switch to credit and debit cards, squeezing the parallel economy of untaxed cash transactions and giving more citizens access to digital financial services.

In a surprise television announcement last November, Modi announced the demonetisation of 500 and 1,000 rupee notes (around £6 and £12), wiping out 85% of the country’s circulating currency overnight.

Two days later, when the banks reopened, long queues snaked around almost every branch, with millions lining up to open bank accounts for the first time. Many used their 12-digit Aadhaar number, linked to their biometric profile, to sign up. Within three weeks, 3m bank accounts had been opened using fingerprint verification, according to estimates.

The moment marked a radical change for India’s banking system, under which applicants were traditionally required to file photocopies of passports or voter IDs. Banks could take weeks, sometimes months, to verify them. Now applicants’ encrypted biometric data can be sent to the Unique Identification Authority of India (UIDAI), a government agency, to be matched against their Aadhaar data, re-encrypted and sent back to the bank.

Despite technical teething problems, the system is designed to allow very fast authorisation. “All this happens in a matter or two or three seconds,” explains Ajay Bhushan Pandey, UIDAI’s director general.

For Pandey, the benefits are clear: paper documents are easy to forge and hard to verify, especially in India where until recently thousands of people still used handwritten passports. Not so biometric data.

Privacy fears

Pandey emphasises that private banks and companies aren’t able to access the entire Aadhaar database, only to use the government interface, which allows them to verify identities.

Nonetheless, many Indians are worried about the privacy implications. Sumandro Chattapadhyay, a director at the Centre for Internet and Society thinktank, is one of them.

For starters, says Chattapadhyay, the law governing use of the biometric database, fast-tracked through parliament last year, is flimsy when it comes to the private sector. Since India lacks a general privacy or data protection law, this leaves corporate use of Aadhaar services effectively unregulated, he says.

This is particularly worrying, says Chattapadhyay, because of the data-sharing possibilities opened up by Aadhaar. It makes it easier for companies not only to share information on individuals’ consumption and mobility habits, but also to link this data up with public records like the electoral register, he says. “Both lead to significant threats to privacy of individuals.”

Chattapadhyay’s fear is that private companies could eventually gain access to government-held personal data, such as income or medical records, while the government could use company data like phone records to target specific individuals in political campaigns.

Already companies are linking Aadhaar numbers with collected metadata. Credit-checking startup CreditVidya, for example, identifies clients using their biometric ID in combination with their internet browsing history and other data, to assign credit scores for users who have no record of loan repayments. Banks then store this processed metadata, for example whether or not someone’s Facebook name is consistent with the name on their bank account.

Its founder Abhishek Agarwal admits there are risks for users: “[I]f someone managed to hack the bank’s security system, as well as the Aadhaar database, they could potentially be able to link your Facebook or LinkedIn data with your biometric information.” But he says this would be hard to do.

Pandey insists the companies are carefully vetted before they can use Aadhaar authentication. But, like Agarwal, he acknowledges the system can never be 100% secure: ““I wouldn’t say it is impossible to break the system, but it is very, very difficult.”

For more details visit http://editors.cis-india.org/internet-governance/news/vidhi-doshi-fingerprint-payments-prompt-privacy-fears-in-india-the-guardian

Unpacking video-based surveillance in New Delhi

Aayush Rathi and Ambika Tandon — 2019-06-20T05:13:25Z

Aayush Rathi and Ambika Tandon presented at an international workshop on 'Urban Data, Inequality and Justice in the Global South', on 14 June 2019, at the University of Manchester. The agenda for the workshop and the slides from the presentation by Aayush and Ambika are available below.

Agenda of the workshop: Download (DOCX)

Slides from the presentation: Download (PDF)

The aim of the workshop was to present findings from case studies on urban data justice commissioned by the Sustainable Consumption Institute and Centre for Development Informatics at the University of Manchester, on aspects of justice in data systems in cities across the world. Aayush and Ambika presented their study on video-based surveillance in New Delhi, which was conducted across a period of 3 months earlier this year. The study aimed to assess the extent to which CCTV surveillance systems in Delhi support the needs of women in the city, including lower class women and those from informal settlements. The study will be published as a working paper by the University of Manchester in the coming months.

For more details visit http://editors.cis-india.org/raw/unpacking-video-based-surveillance-in-new-delhi-urban-data-justice

Understanding Aadhaar and its New Challenges, May 26-27, 2016

sumandro — 2016-05-26T10:29:43Z

A workshop on “Understanding Aadhaar and its New Challenges” is being organised by the Centre for Studies in Science Policy, Jawaharlal Nehru University, and the Centre for Internet and Society, during May 26-27. It is also supported by the Centre for Communication Governance at NLU Delhi, Free Software Movement of India, Knowledge Commons, PEACE, and Center for Advancement of Public Understanding of Science & Technology. This is a legal and technical workshop to be attended by various key researchers and practitioners to discuss the current status of the implementation of the project, in the context of the passing of the Act and the various ongoing cases.

Workshop Programme

First Day, May 26

9:00-9:30	Registration
9:30-10:00	Prof. Dinesh Abrol - Welcome Self-introduction and expectations of participants Dr. Usha Ramanathan - Overview of the Workshop
10:00-11:00	Current Status of Aadhaar Dr. Usha Ramanathan, Legal Researcher, New Delhi - What the 2016 Law Says, and How it Came into Being S. Prasanna, Advocate, New Delhi - Status and Force of Supreme Court Orders on Aadhaar Discussion
11:00-11:30	Tea Break
11:30-13:30	Direct Benefits Transfers Prof. Reetika Khera, Indian Institute of Technology, Delhi - Welfare Needs Aadhaar like a Fish Needs a Bicycle Prof. Ram Kumar, Tata Institute of Social Sciences, Mumbai - Aadhaar and the Social Sector: A critical analysis of the claims of benefits and inclusion Ashok Rao, Delhi Science Forum - Cash Transfers Study Discussion
13:30-14:30	Lunch
14:30-16:00	Aadhaar: Science, Technology, and Security Prof. Subashis Banerjee, Deptt of Computer Science & Engineering, IIT, Delhi - Privacy and Security Issues Related to the Aadhaar Act Pukhraj Singh, former National Cyber Security Manager, Aadhaar, New Delhi - Aadhaar: Security and Surveillance Dimensions Discussion
16:00-16:30	Tea Break
16:30-17:30	Aadhaar - International Dimensions Prof. Chinmayi Arun, Center for Communication Governance, National Law University, Delhi - Biometrics and Mandatory IDs in other parts of the world Dr. Gopal Krishna, Citizens Forum for Civil Liberties - International Dimensions of Aadhaar Discussion
17:30-18:00	High Tea
18:00-19:00	Video Presentations

Second Day, May 27

9:30-11:00	Privacy, Surveillance, and Ethical Dimensions of Aadhaar Prabir Purkayastha, Free Software Movement of India, New Delhi - Surveillance Capitalism and the Commodification of Personal Data Arjun Jayakumar, SFLC - Surveillance Projects Amalgamated Col Mathew Thomas, Bengaluru - The Deceit of Aadhaar Discussion
11:00-11:30	Tea Break
11:30-10:30	Aadhaar: Broad Issues - I Prof. G Nagarjuna, Homi Bhabha Center for Science Education, Tata Institute of Fundamental Research, Mumbai - How to prevent linked data in the context of Aadhaar Dr. Anupam Saraph, Pune - Aadhaar and Moneylaundering Discussion
13:00-13:30	Video Presentations
13:30-14:30	Lunch
14:30-15:30	Aadhaar: Broad Issues - II Prof. MS Sriram, Visiting Faculty, Indian Institute of Management, Bangalore - Financial lnclusion Nikhil Dey, MKSS, Rajasthan (TBC) - Field witness: Technology on the Ground Prof. Himanshu, Centre for Economic Studies & Planning, JNU - UID Process and Financial Inclusion Discussion
15:30-16:00	Conclusion

For more details visit http://editors.cis-india.org/internet-governance/events/understanding-aadhaar-and-its-new-challenges-may-26-27-2016

To be Counted When They Count You: Words of Caution for the Gender Data Revolution

noopur — 2022-02-01T01:06:08Z

In 2015, after the announcement of the SDGs or Sustainable Development Goals, a new global developmental framework through the year 2030, the United Nations described data as the “lifeblood of decision-making and the raw material for accountability” for the purpose of realizing these developmental goals. This curious yet key link between these new developmental goals and the use of quantitative data for agenda setting invited a flurry of big data-led initiatives such as but not limited to Data2X, that sought to further strengthen and solidify the relationship between ‘Big Development’ and ‘Big Data.’

One of those SDG goals (Goal 5) prioritizes gender equality and empowerment of women and girls not only as a standalone goal but also as a crucial factor to realizing the other goals. In response, several academic and non-profit initiatives have begun to interpret and conduct data-led gendered development or the “gender data revolution”. As with other data discourses, the gender-data discourse is also one of ‘speed’, charging ahead using a variety of quantitative and visualization approaches to reveal and eventually solve gendered problems of development.

These interventions also invite some classical critical questions: who is setting the agenda for the gender data revolution and who are its imagined subjects? How are questions of participation and asymmetries of power in developmental research being addressed? How does the gender data revolution address the situatedness as well as incompleteness of data records in the Global South (where most sites of intervention are)? Speaking specifically to the theme of this special issue (‘cross-cultural feminist technologies’), this paper demonstrates how the welfarist discourse of data-led gender development is, in fact, assembled through the overwhelming enumeration of female-identifying bodies in the Global South.

The paper offers critical historical insights from the fields of international development, anthropology, and postcolonial history to caution against both, the possible harms of gender disaggregated datafication as well as the consequences of non-participatory datafication of women, the subjects of the gender data revolution.

Read the full paper here.

This study was undertaken as part of the Big Data for Development network supported by the International Development Research Centre, Canada, and is shared under Creative Commons Attribution 4.0 International license.

The views and opinions expressed on this page are those of their individual authors. Unless the opposite is explicitly stated, or unless the opposite may be reasonably inferred, CIS does not subscribe to these views and opinions which belong to their individual authors. CIS does not accept any responsibility, legal or otherwise, for the views and opinions of these individual authors. For an official statement from CIS on a particular issue, please contact us directly.

For more details visit http://editors.cis-india.org/raw/to-be-counted-when-they-count-you-words-of-caution-for-the-gender-data-revolution

The Technology behind Big Data

Geethanjali Jujjavarapu and Udbhav Tiwari — 2016-12-04T09:53:43Z

The authors undertakes a high-level literature review of the most commonly used technological tools and processes in the big data life cycle. The big data life cycle is a conceptual construct that can be used to study the various stages that typically occur in collecting, storing and analysing big data, along with the principles that can govern these processes.

Download the Paper (PDF, 277 kb)

Introduction

Defining big data is a disputed area in the field of computer science^{^[1]}, there is some consensus on a basic structure to its definition^{^[2]}. Big data is data that is collected in the form of datasets that has three main criteria: size, variety & velocity, all of which operate at an immense scale^{^[3]}. It is ‘big’ in size, often running into petabytes of information, has vast variety within its components, and is created, captured and analysed at an incredibly rapid velocity. All of this also makes big data difficult to handle using traditional technological tools and techniques.

This paper will attempt to perform a high-level literature review of the most commonly used technological tools and processes in the big data life cycle. The big data life cycle is a conceptual construct that can be used to study the various stages that typically occur in collecting, storing and analysing big data, along with the principles that can govern these processes. The big data life cycle consists of four components, which will also be the key structural points of the paper, namely: Data Acquisition, Data Awareness, Data Analytics & Data Governance.⁴ The paper will focus on the aspects that the author believes are relevant for analysing the technological impact of big data on both technology itself and society at large.

Scope: The scope of the paper is to study the technology used in big data using the "Life Cycle of Big Data" as model structure to categorise & study the vast range of technologies that are involved in big data. However, the paper will be limited to the study of technology related directly to the big data life cycle. It shall specifically exclude the use/utilisation of big data from its scope since big data is most often being fed into other, unrelated technologies for consumption leading to rather limitless possibilities.

Goal: Goal of the paper is twofold: a.) to use the available literature on the technological aspects of big data, to perform a brief overview of the technology in the field and b.) to frame the relevant research questions for studying the technology of big data and its possible impact on society.

Data Acquisition

Acquiring big data has two main sub components to it, the first being sensing the existence of the data’ itself and the second, the stage of collecting and storing this data. Both of these subcomponents are incredibly diverse fields, with lots of rapid change occurring in the technology utilised to carry out these tasks. The section will provide a brief overview of the subcomponents and then discuss the technology used to fulfil the tasks.

Data Sensing

Data does not exist in a vacuum and is always created as a part of a larger process, especially in the aspect of modern technology. Therefore, the source of the data itself plays a vital role in determining how it can be captured and analysed in the larger scheme of things. Entities constantly emit information into the environment that can be utilised for the purposes of big data, leading to two main kinds of data: data that is “born digital” or “born analogue.”^{^[4]}

Born Digital Data

Information that is “born digital,” is created, by a user or by a digital system, specifically for use by a computer or data‐processing system. This is a vast range of information and newer fields are being added to this category on a daily basis. It includes, as a short, indicative list: email and text messaging, any form of digital input, including keyboards, mouse interactions and touch screens, GPS location data, data from daily home appliances (Internet of Things), etc. All of this data can be tracked and tagged to users as well as be aggregated to form a larger picture, massively increasing the scope of what may constitute the ‘data’ in big data.

Some indicative uses of how such born digital data is catalogued by technological solutions on the user side, prior to being sent for collection/storage are:

a.) Cookies - There are small, often just text, files that are left on user devices by websites in order to that visit, task or action (for example, logging into an email account) with a subsequent event.^{^[5]} (for example, revisiting the website)

b.) Website Analytics^{^[6]} - Various services, such as Google Analytics, Piwik, etc., can use JavaScript and other web development languages to record a very detailed, intimate track of a user's actions on a website, including how long a user hovers above a link, the time spent on the website/application and in some cases, even the time spent specific aspects of the page.

c.) GPS^{^[7]} - With the almost pervasive usage of smartphones with basic location capabilities, GPS sensors on these devices are used to provide regular, minute driven updates to applications, operating systems and even third parties about the user's location. Modern variations such as A-GPS can be used to provide basic positioning information even without satellite coverage, vastly expanding the indoor capabilities of location collection.

All of these instances of sensing born digital data are common terms, used in daily parlance by billions of people from all over the world, which is a symbolic of just how deeply they have pervaded into our daily lifestyle. Apart from privacy & security concerns this in turn also leads to an exponential increase in the data available to collect for any interested party.

Sensor Data

Information is said to be “analogue” when it contains characteristics of the physical world, such as images, video, heartbeats, etc. Such information becomes electronic when processed by a “sensor,” a device that can record physical phenomena and convert it into digital information. Some examples to better illustrate information that is born analogue but collected via digital means are:

a.) Voice and/or video content on devices - Apart from phone calls and other forms communication, video and voice based interactions have started to regularly be captured to provide enhanced services. These include Google Now^{^[8]}, Cortana^{^[9]} and other digital assistants as well as voice guided navigation systems in cars, etc.

b.) Personal health data such as heartbeats, blood pressure, respiration, velocity, etc. - This personal, potentially very powerful information is collected by dedicated sensors on devices such as Fitbit^{^[10]}, Mi Band^{^[11]}, etc. as well as by increasingly sophisticated smartphone applications such as Google Fit that can do so without any special device.

c.) Camera on Home Appliances - Cameras and sensors on devices such as video game consoles (Kinect^{^[12]} being a relevant example) can record detailed human interactions, which can be mined for vast amounts of information apart from carrying out the basic interactions with the devices itself.

While not as vast a category as born digital data, the increasingly lower costs of technology and ubiquitous usage of digital, networked devices is leading to information that was traditionally analogue in nature to be captured for use at a rapidly increasing rate.

Data Collection & Storage

Traditional data was normally processed using the Extract, Transform, Load (ETL) methodology, which was used to collect the data from outside sources, modify the data to fit needs, and then upload the data into the data storage system for future use.^{^[13]} Technology such as spreadsheets, RDBMS databases, Structured Query Languages (SQL), etc. were all initially used to carry out these tasks, more often than not manually. ^{^[14]}

However, for big data, the methodology traditionally followed is both inefficient and insufficient to meet the demands of modern use. Therefore, the Magnetic, Agile, Deep (MAD) process is used to collect and store data^{^[15]}^{^[16]}. The needs and benefits of such a system are: attracting all the data sources regardless of their quality (magnetic), logical and physical contents of storage systems adapting to the rapid data evolution in big data (agile) and complex algorithmic statistical analysis required of big data on a very short notice^{^[17]}. (deep)

The technology used to perform data storage using the MAD process requires vast amount of processing power, which is very difficult to create in a single, physical space/unit for nonstate or research entities, who cannot afford supercomputers. Therefore, most solutions used in big data rely on two major components to store data: distributed systems and Massive Parallel Processing^{^[18]} (MPP) that run on non-relational (in-memory) database systems. Database performance and reliability is traditionally gauged using pure performance metrics (FLOPS per second, etc.) as well as the Atomicity, consistency, isolation, durability (ACID) criteria.^{^[19]} The most commonly used database systems for big data applications are given below. The specific operational qualities and performance of each of these databases is beyond the scope of this review but the common criteria that makes them well suited for big data storage have been delineated below.

Non-relational databases

Databases traditionally used to be structured entities that operated solely on the ability to correlate information stored in them using explicitly defined relationships. Even prior to the advent of big data, this outlook was turning out to be a limiting factor in how large amounts of stored information could be leveraged, this led to the evolution of non relational database systems. Before going into them in detail, a basic primer on their data transfer protocols will be helpful in understanding their operation.

A protocol is a model that structures instructions in a particular manner so that it can be reproduced from one system to another^{^[20]}^{^[21]}. The protocols which govern technology in the case of big data have gone through many stages of evolution, starting off with simple HTML based systems^{^[22]}, which then evolved to XML driven SOAP systems^{^[23]}, which led to JavaScript Object Notation, or JSON^{^[24]}, the currently used form for in most big database systems. JSON is an open format used to transfer data objects, using human-readable text and is the basis for most of the commonly used non-relational database management systems. Examples of Non-relational databases also known as NoSQL databases, include MongoDB^{^[25]}, Couchbase^{^[26]}, etc. They were developed for both managing as well as storing unstructured data. They aim for scaling, flexibility, and simplified development. Such databases rather focus on the high-performance scalable data storage, and allow tasks to be written in the application layer instead of databases specific languages, allowing for greater interoperability.^{^[27]}

In-Memory Databases

In order to overcome performance limitation of traditional database systems, some modern databases now use in-memory databases. These systems manage the data in the RAM memory of the server, thus eliminating storage disk input/output. This allows for almost realtime responses from the database, in comparisons to minutes or hours required on traditional database systems. This improvement in the performance is so massive that, entirely new applications are being developed for using IMDB systems.^{^[28]} These IMDB systems are also being used for advanced analytics on big data, especially to increase the access speed to data and increase the scoring rate of analytic models for analysis.^{^[29]} Examples of IMDB include VoltDB^{^[30]}, NuoDB^{^[31]}, SolidDB^{^[32]} and Apache Spark^{^[33]}.

Hybrid Systems

These are the two major systems used to store data prior to it being processed or analysed in a big data application. However, the divide between data storage and data management is a slim one and most database systems also contain various unique attributes that cater them to specific kinds of analysis. (as can be seen from the IMDB example above) One example of a very commonly used Hybrid system that deals with storage as well as awareness of the data is Apache Hadoop³³, which is detailed below.

Apache Hadoop

Hadoop consists of two main components: the HDFS for the big data storage, and MapReduce for big data analytics, each of which will be detailed in their respective section.

The HDFS^{^[34]}^{^[35]} storage function in Hadoop provides a reliable distributed file system, stored across multiple systems for processing & redundancy reasons. The file system is optimized for large files, as single files are split into blocks and spread across systems known as cluster nodes.^{^[36]} Additionally, the data is protected among the nodes by a replication mechanism, which ensures availability even if any node fails. Further, there are two types of nodes: Data Nodes and Name Nodes.^{^[37]} Data is stored in the form of file blocks across the multiple Data Nodes while the Name Node acts as an intermediary between the client and the Data Node, where it directs the requesting client to the particular Data Node which contains the requested data.

This operating structure for storing data also has various variations within Hadoop such as HBase for key/value pair type queries (a NoSQL based system), Hive for relational type queries, etc. Hadoop’s redundancy, speed, ability to run on commodity hardware, industry support and rapid pace of development have led to it being almost co-equivalently associated with big data.^{^[38]}

Data Awareness

Data Awareness, in the context of big data, is the task of creating a scheme of relationships within a set of data, to allow different users of the data to determine a fluid yet valid context and utilise it for their desired tasks.^{^[39]} It is a relatively new field, in which most of the work is currently being done on semantic structures to allow data to gain context in an interoperable format, in contrast to the current system where data is given context using unique, model specific constructs.^{^[40]} (such as XML Schemes, etc.)

Some of the original work on this field was carried out in the form of utilising the Resource Description Framework (RDF), which was built primarily to allow describing of data in a portable manner, especially being agnostic towards platforms and systems for Semantic Web at the W3C. SPARQL is the language used to implement RDF based designs but both largely remain underutilised in both the public domain as well as big data. Authors such as Kurt

Cagle^{^[41]} and Bob DuCharme^{^[42]} predict its explosion in the next couple of years. Companies have also started realising the value of interoperable context, with Oracle Spatial^{^[43]} and IBM’s DB2^{^[44]} already including RDF and SPARQL support in the past 3 years.

While underutilised, the rapid developments taking place in the field will make the impact that data awareness may have on big data as big as Hadoop and maybe even SQL. Some aspects of it are already beginning to be used in Artificial Intelligence, Natural Language Processing, etc. with tremendous scope for development.^{^[45]}

Data Processing & Analytics

Data Processing largely has three primary goals: a. determines if the data collected is internally consistent; b. make the data meaningful to other systems or users using either metaphors or analogy they can understand; and (what many consider most importantly) provide predictions about future events and behaviours based upon past data and trends.^{^[46]}

Being a very vast field with rapidly changing technologies governing its operation, this section will largely concentrate on the most commonly used technologies in data analytics.

Data analytics requires four primary conditions to be met in order to carry out effective processing: fast, data loading, fast query processing, efficient utilisation of storage and adaptivity to dynamic workload patterns. The analytical model most commonly associated with meeting this criteria and with big data in general is MapReduce, detailed below. There are other, more niche models and algorithms (such as Project Voldemort^{^[47]} used by LinkedIn), which are used in big data but they are beyond the scope of the review, and more information about them can be read at article linked in the previous citation. (Reference architecture and classification of technologies, products and services for big data system)

MapReduce

MapReduce is a generic parallel programming concept, derived from the “Map” and “Reduce” of functional programming languages, which makes it particularly suited for big data operations. It is at the core of Hadoop^{^[48]}, and performs the data processing and analytics functions in other big data systems as well.^{^[49]} The fundamental premise of MapReduce is scaling out rather than scaling up, i.e., (adding more numerical resources, rather than increasing the power of a single system)^{^[50]}

MapReduce operates by breaking a task down into steps and executing the steps in parallel, across many systems. This comes with two advantages, a reduction in the time needed to finish the task and also a decrease in the amount of resources one has to expend to perform the task, in both power and energy. This model makes it ideally suited for the large data sets and quick response times required of big data operations generally.

The first step of a MapReduce job is to correlate the input values to a set of keys/value pairs as output. The “Map” function then partitions the processing tasks into smaller tasks, and assigns them to the appropriate key/value pairs.^{^[51]} This allows unstructured data, such as plain text, to be mapped to a structured key/value pair. As an example, the key could be the punctuation in a sentence and the value of the pair could be the number of occurrences of the punctuation overall. This output of the Map function is then passed on “Reduce” function.^{^[52]} Reduce then collects and combines this output, using identical key/value pairs, to provide the final result of the task.^{^[53]} These steps are carried using the Job Tracker & Task Tracker in Hadoop but different systems have different methodologies to carry out similar tasks.

Data Governance

Data Governance is the act of managing raw big data as well as the processed information that arises from big data in order to meet legal, regulatory and business imposed requirements. While there is no standardized format for data governance, there have been increasing call with various sectors (especially healthcare) to create such a format to ensure reliable, secure and consistent big data utilisation across the board. The following tactics and techniques have been utilised or suggested for data governance, with varying degrees of success:

Zero-knowledge systems: This technological proposal maintains secrecy with respect to the low-level data while allowing encrypted data to be examined for certain higherlevel abstractions.^{^[54]} For the system to be zero-knowledge, the client’s system will have to encrypt the data and send it to the storage provider. Due to this, the provider stores the data in the encrypted format and cannot decipher the same unless he/she is in possession of the key which will decrypt the data into plaintext. This allows the individual to store his data with a storage provider while also maintaining anonymity of the details contained in such information. However, these are currently just beginning to be used in simple situations. As of now, they are not expandable to unstructured and complex cases and have to be developed marginally before they can be used for research and data mining purposes.
Homomorphic encryption: Homomorphic encryption is a privacy preserving technique which performs searches and other computations over data that is encrypted while also protecting the individual’s privacy.^{^[55]} This technique has however been considered to be impractical and is deemed to be an unlikely policy alternative for near future purposes in the context of preserving privacy in the age of big data.^{^[56]}
Multi-party computation: In this technique, computation is done on encrypted distributed data stores.^{^[57]} This mechanism is closely related to homomorphic encryption where individual data is kept private using encryption algorithms called “collusion-robust” while the same is used to calculate statistics.^{^[58]} The parties involved are aware of some private data and each of them use a protocol which produces results based on the information they are aware of and the information they are not aware of, without revealing the data they are not already aware of.^{^[59]} Multi-party computations thus help in generating useful data for statistical and research purposes without compromising the privacy of the individuals.

Differential Privacy: Although this technological development is related to encryption, it follows a different technique. Differential privacy aims at maximizing the precision of computations and database queries while reducing the identifiability of the data owners who have records in the database, usually through obfuscation of query results.^{^[60]} This is widely applied today in the existence of big data in order to ensure preservation of privacy while trying to reap the benefits of large scale data collection.^{^[61]}
Searchable encryption: Through this mechanism, the data subject can make certain data searchable while minimizing exposure and maximizing privacy.^{^[62]} The data owner can make his information available through search engines by providing the data in an encrypted format but by adding tags consisting of certain keywords which can be deciphered by the search engine. This encrypted data shows up in the search results when searched with these particular keywords but can only be read when the person is in possession of the key which is required for decrypting the information.

This technique of encryption provides maximum security to the individual’s data and preserves privacy to the greatest possible extent.

K-anonymity: The property of k-anonymity is being applied in the present day in order to preserve privacy and avoid re-identification.^{^[63]} A certain data set is said to possess the property of k-anonymity if individual specific data can be released and used for various purposes without re-identification. The analysis of the data should be carried out without attributing the data to the individual to whom it belongs and should give scientific guarantees for the same.
Identity Management Systems: These systems enable the individuals to establish and safeguard their identities, explain those identities with the help of attributes, follow the activity of their identities and also delete their identities if they wish to.^{^[64]} It uses cryptographic schemes and protocols to make anonymous or pseudonymous the identities and credentials of the individuals before analysing the data.
Privacy Preserving Data Publishing: This is a method in which the analysts are provided with the individual’s personal information with the ability to decipher particular information from the database while preventing the inference of certain other information which might lead to a breach of privacy.^{^[65]} Data which is essential for the analysis will be provided for processing while sensitive data will not be disclosed. This tool primarily focuses on microdata.
Privacy Preserving Data Mining: This mechanism uses perturbation methods and randomization along with cryptography in order to permit data mining on a filtered version of the data which does not contain any form of sensitive information. PPDM focuses on data mining results unlike PPDP.^{^[66]}

Conclusion

Studying the technology surrounding big data has led to two major observations: the rapid pace of development in the industry and the stark lack of industry standards or government regulations directed towards big data technologies. These observations have been the primary motivating factor for framing further research in the field. Understanding how to deal with big data technologically, rather than just the potential regulation of possible harms after the technological processes have been performed might be critical for the human rights dialogue as these processes become even more extensive, opaque and technologically complicated.

[1] EMC: Data Science and Big Data Analytics. In: EMC Education Services, pp. 1–508 (2012)

[2] Bakshi, K.: Considerations for Big Data: Architecture and Approaches. In: Proceedings of the IEEE Aerospace Conference, pp. 1–7 (2012)

[3] Adams, M.N.: Perspectives on Data Mining. International Journal of Market Research 52(1), 11–19 (2010) ⁴ Elgendy, N.: Big Data Analytics in Support of the Decision Making Process. MSc Thesis, German University in Cairo, p. 164 (2013)

[4] Big Data and Privacy: A Technological Perspective - President’s Council of Advisors on Science and

Technology (May 2014)

[5] Chen, Hsinchun, Roger HL Chiang, and Veda C. Storey. "Business Intelligence and Analytics: From Big Data to Big Impact." MIS quarterly 36.4 (2012): 1165-1188.

[6] Chandramouli, Badrish, Jonathan Goldstein, and Songyun Duan. "Temporal analytics on big data for web advertising." 2012 IEEE 28th international conference on data engineering. IEEE, 2012.

[7] Laurila, Juha K., et al. "The mobile data challenge: Big data for mobile computing research." Pervasive Computing. No. EPFL-CONF-192489. 2012.

[8] Lazer, David, et al. "The parable of Google flu: traps in big data analysis." Science 343.6176 (2014): 12031205.

[9] ibid

[10] Banaee, Hadi, Mobyen Uddin Ahmed, and Amy Loutfi. "Data mining for wearable sensors in health monitoring systems: a review of recent trends and challenges." Sensors 13.12 (2013): 17472-17500.

[11] ibid

[12] Chung, Eric S., John D. Davis, and Jaewon Lee. "Linqits: Big data on little clients." ACM SIGARCH Computer Architecture News. Vol. 41. No. 3. ACM, 2013.

[13] Kornelson, Kevin Paul, et al. "Method and system for developing extract transform load systems for data warehouses." U.S. Patent No. 7,139,779. 21 Nov. 2006.

[14] Henry, Scott, et al. "Engineering trade study: extract, transform, load tools for data migration." 2005 IEEE Design Symposium, Systems and Information Engineering. IEEE, 2005.

[15] Cohen, Jeffrey, et al. "MAD skills: new analysis practices for big data." Proceedings of the VLDB Endowment

[16] .2 (2009): 1481-1492.

[17] Elgendy, Nada, and Ahmed Elragal. "Big data analytics: a literature review paper." Industrial Conference on Data Mining. Springer International Publishing, 2014.

[18] Wu, Xindong, et al. "Data mining with big data." IEEE transactions on knowledge and data engineering 26.1 (2014): 97-107.

[19] Supra Note 17

[20] Hu, Han, et al. "Toward scalable systems for big data analytics: A technology tutorial." IEEE Access 2 (2014):

[21] -687.

[22] Kurt Cagle, Understanding the Big Data Lifecycle - LinkedIn Pulse (2015)

[23] Coyle, Frank P. XML, Web services, and the data revolution. Addison-Wesley Longman Publishing Co., Inc., 2002.

[24] Pautasso, Cesare, Olaf Zimmermann, and Frank Leymann. "Restful web services vs. big'web services: making the right architectural decision." Proceedings of the 17th international conference on World Wide Web. ACM, 2008.

[25] Banker, Kyle. MongoDB in action. Manning Publications Co., 2011

[26] McCreary, Dan, and Ann Kelly. "Making sense of NoSQL." Shelter Island: Manning (2014): 19-20.

[27] ibid

[28] Zhang, Hao, et al. "In-memory big data management and processing: A survey." IEEE Transactions on Knowledge and Data Engineering 27.7 (2015): 1920-1948.

[29] ibid

[30] ibid

[31] Supra Note 20

[32] Ballard, Chuck, et al. IBM solidDB: Delivering Data with Extreme Speed. IBM Redbooks, 2011.

[33] Shanahan, James G., and Laing Dai. "Large scale distributed data science using apache spark." Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2015. ³³ Shvachko, Konstantin, et al. "The hadoop distributed file system." 2010 IEEE 26th symposium on mass storage systems and technologies (MSST). IEEE, 2010.

[34] Borthakur, Dhruba. "The hadoop distributed file system: Architecture and design." Hadoop Project Website

[35] .2007 (2007): 21.

[36] ibid

[37] ibid

[38] Zikopoulos, Paul, and Chris Eaton. Understanding big data: Analytics for enterprise class hadoop and streaming data. McGraw-Hill Osborne Media, 2011.

[39] Bizer, Christian, et al. "The meaningful use of big data: four perspectives--four challenges." ACM SIGMOD Record 40.4 (2012): 56-60.

[40] Kaisler, Stephen, et al. "Big data: issues and challenges moving forward." System Sciences (HICSS), 2013 46th Hawaii International Conference on. IEEE, 2013.

[41] Supra Note 21

[42] DuCharme, Bob. "What Do RDF and SPARQL bring to Big Data Projects?." Big Data 1.1 (2013): 38-41.

[43] Zhong, Yunqin, et al. "Towards parallel spatial query processing for big spatial data." Parallel and

Distributed Processing Symposium Workshops & PhD Forum (IPDPSW), 2012 IEEE 26th International. IEEE, 2012.

[44] Ma, Li, et al. "Effective and efficient semantic web data management over DB2." Proceedings of the 2008 ACM SIGMOD international conference on Management of data. ACM, 2008.

[45] Lohr, Steve. "The age of big data." New York Times 11 (2012).

[46] Pääkkönen, Pekka, and Daniel Pakkala. "Reference architecture and classification of technologies, products and services for big data systems." Big Data Research 2.4 (2015): 166-186.

[47] Sumbaly, Roshan, et al. "Serving large-scale batch computed data with project voldemort." Proceedings of the 10th USENIX conference on File and Storage Technologies. USENIX Association, 2012.

[48] Bar-Sinai, Michael. "Big Data Technology Literature Review." arXiv preprint arXiv:1506.08978 (2015).

[49] ibid

[50] Condie, Tyson, et al. "MapReduce Online." Nsdi. Vol. 10. No. 4. 2010.

[51] Supra Note 47

[52] Dean, Jeffrey, and Sanjay Ghemawat. "MapReduce: a flexible data processing tool." Communications of the ACM 53.1 (2010): 72-77.

[53] ibid

[54] Big Data and Privacy: A Technological Perspective, White House,

https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_big_data_and_privacy__may_2014

[55] Tene, Omer, and Jules Polonetsky. "Big data for all: Privacy and user control in the age of analytics." Nw. J. Tech. & Intell. Prop. 11 (2012): xxvii.

[56] Big Data and Privacy: A Technological Perspective, White House,

https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_big_data_and_privacy__may_2014

[57] Privacy by design in big data, ENISA

[58] Big Data and Privacy: A Technological Perspective, White House,

https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_big_data_and_privacy__may_2014

[59] Id

[60] Id

[61] Tene, Omer, and Jules Polonetsky. "Privacy in the age of big data: a time for big decisions." Stanford Law Review Online 64 (2012): 63.

[62] Lane, Julia, et al., eds. Privacy, big data, and the public good: Frameworks for engagement. Cambridge University Press, 2014.

[63] Crawford, Kate, and Jason Schultz. "Big data and due process: Toward a framework to redress predictive privacy harms." BCL Rev. 55 (2014): 93.

[64] http://homes.esat.kuleuven.be/~sguerses/papers/DanezisGuersesSurveillancePets2010.pdf

[65] Seda Gurses and George Danezis, A critical review of 10 years of privacy technology, August 12th 2010, http://homes.esat.kuleuven.be/~sguerses/papers/DanezisGuersesSurveillancePets2010.pdf

[66] Id

For more details visit http://editors.cis-india.org/internet-governance/blog/technology-behind-big-data