BIG DATA ANALYTICS

Big data is the ocean of information  – vast zeta bytes of data flowing from our computers, mobile devices and machine sensors through archives, docs, business apps, social media, public web, machine log data etc.


The rapid growth in industrialization and technology has paved way for seamless flow of data across the globe. Every endpoint device has become a content creation and capture device that has enabled faster and more efficient business processes while also driving massive unstructured data growth. Big data analytic solutions with the help of Internet of things different providers  provides platform that allow you to analyze massive volumes of data, which in turn leads to unprecedented business intelligence you need to make faster and profitable decisions. This plays an indispensable role in predictive analysis by revealing the latent insights on future trends, allowing the client to build a predictive model that brings in better visualization to practice the most economical means to achieve profitable outcomes.


The power of big data can be explored to design predictive models that provide real time business intelligence and collective insights about future trends, thereby enabling the enterprises to make more flexible and dynamic decisions and adopt best business strategies that help them run better. Applied to business, predictive analysis helps to compare the statistical measure of the profit, performance and growth by analyzing the current and the historic data and also equip them to face challenges and risks by early anticipation.

Green Energy Companies Need of Analytics

Our world is becoming more and more connected. Data, machines, people and processes are interlinked in ways we could only imagine before. With 15 billion web-enabled devices and an emerging global middle class poised to exceed five billion, transaction, machine-to-machine, and conversation data is doubling worldwide every 18 months.

While this can be daunting to think about,  immense, untapped value in this data – and a huge opportunity to translate that value into tangible business outcomes. By accessing the data and making it fit for analytic use, we help our customers identify trends that reveal new markets, channels, and innovations. The bottom line: Data and its corresponding insight can be catalysts for employee empowerment, decision making, and an increased reach and relevance in the marketplace.

An Untapped Opportunity
Many organizations remain challenged on how to capitalize on the value of their data. The most common obstacles include:
Not enough access to needed data  - It’s hard to manage and consume all available information when there is so much of it. Using only the most accessible information can mean missing new signals.
Too many systems - Many organizations rely on a patchwork of products, approaches, and methodologies. Managing such a mashup can quickly become unwieldy and expensive – and lead to suboptimal decision making.
Outdated BI tools - Many key reports are based on week- or month-old data, when the pace of business demands real-time analysis encompassing predictive analytics, visualization tools, and data discovery.

The Business Value Cycle
To stay competitive you need to overcome the obstacles and derive the maximum value possible from your data. The most successful organizations document their value creation process using analytics to inform both strategic initiatives and tactical operations. We refer to this as the Business Value Cycle discover, plan, inform, and anticipate.

Analytics in Action
Companies who leverage the full potential of analytics can boost efficiency, enter new markets and squeeze margin from existing businesses.green energy company can bring value while keeping maintenance and operations costs at current rates,new customer markets while maintaining their traditional power customers. Analyzing its social media data determine brand sentiment and monitor fluctuations as the rebranding initiative evolve,can also predict field equipment, turbine, and solar panel maintenance based on meter and usage data,can also track high-value assets and truck movements for inefficient usage or abnormal behavior which helps cut costs and enables investment into new markets.

Using analytics and the business value cycle, Green energy companies can discover and assess the risks for their business transformation, devise a strategy and plan, inform the organization, and gain valuable insight with real-time reporting to anticipate and predict the potential outcomes. In addition, they can improve many aspects for their business using analytics that include:

Regulatory compliance-With governance, risk and compliance software,  can monitor hanging regulations to ensure compliance as they enter new markets.
Business transformation planning and costs-can plan their budgets, consolidate and report on their subsidiary performance and the impact of their new corporate initiatives.
Brand sentiment-Using business intelligence the marketing department can capture, analyze and visualize their social media comments to improve customer experience and optimize campaign performance.
Predictive maintenance-The field operations team can analyze a continuous stream of machine data and diagnostics to predict performance degradation and significantly minimize equipment downtime to ensure customer satisfaction.

Big Data online courses

Here are some of the best big data classes offered online:

Udemy
Big Data and Hadoop Essentials

The course is perfect for people who wants to begin a career in Big data technologies. Udemy, a one of the leading virtual schools, brushes the fundamentals to understand the cryptic Big data problems and how Hadoop can solve them. The best thing about this class? It is free!

Coursera
Web Intelligence and Big Data

Coursera's take on Big data course focuses on web-intelligence applications essential in social media, mobile devices and sensors. This is based on the map-reduce parallel programming pattern. It is also anchored on distributed file systems, no-SQL database and stream computing engines. This 9-week long class is offered as a self learning course, which does not require superior essay and exam skills.

Duke University via Coursera
Data Analysis & Statistical Inference

Duke university in partnership with Coursera lets you delve more into the wonders of Big data technology. In this 10-week course, you'll have a deeper understanding of data collection, and data limitation methods and it affects range of interference. You will also have hands-on training on estimation and testing methods to test single variables that helps you make data-based decisions. This course is for free.

Harvard Extension via iTunes U
Massively Parallel Computing

Big data can quickly crush centralized computing approaches. That is why learning about massively computing is a very helpful big data skill. This hands-on course under the famous name of Harvard School of Engineering and Applied Sciences, covers parallel programming models, multi-thread programming, cluster and cloud computing, and MapReduce through Hadoop.

Code School
Learn The R Programming Language

The class is divided into eight levels to better understand R programming, for data analysis and visualization, great from statistical computing and images. Every class gives you a deep and practical knowledge on the use of R language. Though the course is free, and focuses on fundamentals rather than advance lessons, its interactive format makes the lessons worthwhile.  

MIT Open Courseware
Advanced Data Structures

Massachusetts Institute of Technology offers an advance lesson on data structures and its important application on Algorithms, like that of Google. The study also talks about current directions of research and findings in data structure. The course, however requires you to pass an undergraduate algorithm class, or the open Courseware's Design and Analysis of Algorithms class.

Smarter Utilities for Smarter Cities India's Perspecitve

Adoption of smart grids or distribution automation is the need of the hour.
Solar power in the form of distributed rooftop generation will have a significant role in providing smart cities with an environmentally sustainable power supply. Distribution automation can provide an excellent way to integrate power from renewable and traditional sources, by scheduling each energy source to optimise value.
It also has the possibility to identify customer outages before individual customers call the utility to report them.

Smart water utilities, new automated systems that can interpret data from factors such as water source, pump efficiency to estimate customer usage and making billing more efficient and accurate.

Internet of Things remote meter reading ,share communications infrastructure. are examples of where utilities could collaborate to leverage the investment costs between utilities implementing similar functions. Network convergence lowers the operational and maintenance costs of supporting mission critical infrastructure, while providing scalability.

Application of data analytics and management programs will allow utility managers to make smarter operating decisions and efficiently deliver on the business goals of the utility enterprise.

Improving the efficiency of utilities, which in turn would make communities more adaptive, resilient and sustainable, is the key to development of 'Smart Cities'. With a smartphone in the hand of every Indian in near futuer, the prospect of a cloud-controlled infrastructure platform would pave the way for every urban citizen to utilise resources in the most optimal manner, heralding an era of a smart, responsible and an environment citizen

Smart Cities - India Perspective

Smart Cities – Indian Perspective                   

India is a country of rich physical and human resources. It has flourishing green fields and green farms, high rising mountains which hover over our cities, long elegant rivers which keep the nations watered and plains, never-ending natural resources etc. In short, it can be said that that a mini world residing in India.

The rise in population, unconventional use or wastage of natural resources and rising corruption has led in many of our countrymen losing trust in the nation’s true prospective. Our countrymen’s vision of an urbanized India is still limited to the hopes of 24 hour running water, continuous power supply and good housing. After nearly 7 decades of an autonomous government rule, the dreams of Indian should have tangled bullet trains, rising economy and a sophisticated technology. But these things still remain a daydream for a common Indian.

Why we need a Smart City:

The attraction of more job opportunities and quality of  service made many of the india people slowly migrating  from Rural areas to cities. In days to come the migration to cities would  be on a vast scale for better living  and opportunities .

As per the study imparted by McKinsey Global Institute, by the year 2030, 70% of jobs and service opportunities will be in the cities. The study also finds that Indian cities will fabricate 70% of the nation’s GDP and will raise the countries per capita income fourfold.

Improved urban population will mean more power spending and complexness in city management. It means that the government will face tough job in dealing with everything from bylaw and order, health and security; to power, waste and transportation management.

All these things make it clear that India should gear up itself to administer this rapid rise in urban population creatively and make certain that the affect of this trend is utilized for the nation’s richness and growth.

Factors necessary for Smart Cities:

The idea of smart cities can be more easily interpreted by some cautionary smart cities around the world. Copenhagen (Denmark), Amsterdam (Netherland), Vienna (Austria), Barcelona (Spain), Paris (France), London (England), Berlin (Germany) etc

•             Improving or protecting the environment is one of the main aims of a smart city. Say for example Copenhagen has one of the smallest carbon footprints /capita in the world (less than two tons / capita).

•             Traffic reduction and managing is also a fundamental element in a smart city. In Amsterdam, 67% of all trips are done by cycling or walking.

•             Paris is also famous for their grand and broadly used bikesharing network which has led to a 5% reduction in vehicle congestion.

•             Use of solar energy for the 100 % power generation is also an important factor for a smart city.

Role of Technology in Developing Smart City:

The idea of a smart city is a moderately new one. Cities in the urbanized world are developing technology master plans and then using these plans to develop a citywide authority and control network that supervises and optimizes the delivery of services like power, water, traffic and healthcare. The fundamental principle of a smart city is making infrastructure network and release of services more capable across, logistics, water supply, telecommunication and gas supply.

Indian cities, in a small way, are using sophisticated technology within sections to solve problems. These include traffic control, by means of sensors to monitor water leaks, chasing garbage trucks through GPS to guarantee they put their waste at chosen landfills, energy management in smart buildings and complexes. Also under progress are smart townships that are prohibited centrally, and entire cities along the Delhi-Mumbai Industrial Corridor.

Typically in a smart city, sensors will allow real-time inputs to a control centre on fresh water, energy, civic transport, communal safety, edification, and healthcare. Intelligent communication tools will let executives manage and react to emergencies quickly as well as provide residents with steady real-time inputs.

Role of IoT (Internet of Things )Technology in Developing Smart City:

The Indian approach for Smart Cities:

The cities with constant or projected 100 smart cities include surat, delhi, vizag  etc. Many of these cities will comprise special investing areas or special economic zones with customized policy and tax structures to make it eye-catching for foreign investment.

With numerous reviewed laws and rules for the real estate sector in India, the above strategy of the administration will also prove to be a huge advantage for the real estate developers as well as the builders. Because, the construction of smart cities will need the capability of builders and more significantly, the prudence of real estate developers.

There are many ways to make housing, commercial and public spaces sustainable by ways of applied science, but an elevated proportion of the total energy consumption is still in the hands of end users and their doings. For instance, the success of such a city depends on inhabitants, entrepreneurs, visitors and their participation in energy saving and accomplishment of new technologies.

However, it should also be recalled that every smart city has two more main facilitators apart from the main enabler which is technology. The other two significant enablers are: the inhabitants of the city and the management. Even with all the technology a smart city gets, it’s the people and the management that are at the centre of the smart city.

So, a smart city is built by these three facilitators on the following six columns: Smart governance, Smart populace, Smart mobility and move, Smart livelihood and housing, Smart environment and smart economy. If we want smart cities, we should make sure that all the six pillars are significant enough to assume the weight of the stargazed smart city.

City leaders all over the world have bosomed the smart city perception with ebullience. They are acclaiming ground-breaking projects and putting out a vision for how cities can use technology to meet sustainability goals, enhance local economies, and ameliorate services. This promise to changing how cities function is driving the constant interest in smart cities. Moreover, the smart city model is evolving as more cities set out their own schedule and a growing range of suppliers deliver solutions to meet their rising needs.

How to become Data Scientist

past year, interest in data science has soared.Nate Silver is a household name, companies everywhere are searching for unicorns, and professionals in many different disciplines have begun eyeing the well-salaried profession as a possible career move.

In our recruiting searches here at Burtch Works, we’ve spoken to many analytics professionals who are considering adapting their skills to the growing field of data science, and have questions about how to do so. From my perspective as a recruiter, I wanted to put together a list of technical and non-technical skills that are critical to success in data science, and at the top of hiring managers’ lists.

Every company will value skills and tools a bit differently, and this is by no means an exhaustive list, but if you have experience in these areas you will be making a strong case for yourself as a data science candidate.

Technical Skills: Analytics

1. Education – Data scientists are highly educated – 88% have at least a Master’s degree and 46% have PhDs – and while there are notable exceptions, a very strong educational background is usually required to develop the depth of knowledge necessary to be a data scientist. Their most common fields of study are Mathematics and Statistics (32%), followed by Computer Science (19%) and Engineering (16%).

2. SAS and/or R – In-depth knowledge of at least one of these analytical tools, for data science R is generally preferred.

Technical Skills: Computer Science

3. Python Coding – Python is the most common coding language I typically see required in data science roles, along with Java, Perl, or C/C++.

4. Hadoop Platform – Although this isn’t always a requirement, it is heavily preferred in many cases. Having experience with Hive or Pig is also a strong selling point. Familiarity with cloud tools such asAmazon S3 can also be beneficial.

5. SQL Database/Coding – Even though NoSQL and Hadoop have become a large component of data science, it is still expected that a candidate will be able to write and execute complex queries in SQL.

6. Unstructured data – It is critical that a data scientist be able to work with unstructured data, whether it is from social media, video feeds or audio.

Non-Technical Skills

7. Intellectual curiosity – No doubt you’ve seen this phrase everywhere lately, especially as it relates to data scientists. Frank Lo describes what it means, and talks about other necessary “soft skills” in his guest blog posted a few months ago.

8. Business acumen – To be a data scientist you’ll need a solid understanding of the industry you’re working in, and know what business problems your company is trying to solve. In terms of data science, being able to discern which problems are important to solve for the business is critical, in addition to identifying new ways the business should be leveraging its data.

9. Communication skills – Companies searching for a strong data scientist are looking for someone who can clearly and fluently translate their technical findings to a non-technical team, such as the Marketing or Sales departments. A data scientist must enable the business to make decisions by arming them with quantified insights, in addition to understanding the needs of their non-technical colleagues in order to wrangle the data appropriately. Check out our recent flash survey for more information on communication skills for quantitative professionals.

The next question I always get is, “What can I do to develop these skills?” There are many resources around the web, but I don’t want to give anyone the mistaken impression that the path to data science is as simple as taking a few MOOCs. Unless you already have a strong quantitative background, the road to becoming a data scientist will be challenging – but not impossible.

However, if it’s something you’re sincerely interested in, and have a passion for data and lifelong learning, don’t let your background discourage you from pursuing data science as a career. Here are a few of the resources we’ve found to be helpful:

Resources

1. Advanced Degree – More Data Science programs are popping up to serve the current demand, but there are also many Mathematics, Statistics, and Computer Science programs.
2. MOOCs –Coursera, Udacity, and codeacademy are good places to start.
3. Certifications – KDnuggets has compiled an extensive list.
4. Bootcamps – For more information about how this approach compares to degree programs or MOOCs, 
5. Kaggle – Kaggle hosts data science competitions where you can practice, hone your skills with messy, real world data, and tackle actual business problems. Employers take Kaggle rankings seriously, as they can be seen as relevant, hands-on project work.
6. LinkedIn Groups – Join relevant groups to interact with other members of the data science community.
7. Data Science Central and KDnuggets – Data Science Central and KDnuggets are good resources for staying at the forefront of industry trends in data science.
8. The Burtch Works Study: Salaries of Data Scientists – If you’re looking for more information about the salaries and demographics of current data scientists be sure to download our data scientist salary study.

Smart Cities in India

Smart City  offers economic activities and employment opportunities to a wide section of its residents, regardless of their level of education, skills or income levels


By 2050, the world will witness a mass exodus of people into cities. 2 out of every 3 people will be living in urban areas which translates into 6.3 Billion urban dwellers. . By 2050, Asia and Africa will account for 86% the world’s urban population. India alone will add more than 400 Million people to its cities – that is twice the population of Brazil today. The consequence of this migration - especially in a fast growing economy like India - is a significant increase in the demand and consumption of resources. The coal reserves in the existing coal mines in India are likely to get exhausted in little more than 50 years at the current rate of consumption. India’s oil imports account for the biggest share in the Current Account Deficit. Cities contribute to 70% of India’s GDP. The growth of cities therefore is inevitable.

However, unplanned growth and distribution of resources could prove to be catastrophic to the economy and impede progress. Building sustainable and Smart cities from scratch and retrofitting sustainability features in already existing cities is the only way out.
Smart Cities are the only perceivable solution to urbanization of this scale.

A Smart city includes a structure that is resource efficient and has a minimal impact on the environment. Among other things, a smart city reduces the energy and water requirement by employing technology and smart construction & design techniques, reduces the generation of solid waste and uses renewable sources to meet energy requirements.

Additionally, to promote a more convenient way of life, a smart city incorporates a sophisticated Information and Communication Infrastructure. A robust transport network to move people is also established.
While retrofitting smart city-like features into an already existing city is a possible solution, it comes with its limitations. Retrofitting is more expensive and inconvenient primarily because of high replacement costs and limitations of the existing structures. The market for retrofitting is still in its nascent stages and therefore not fully understood. While areas like lighting, air conditioning, etc. have seen some technological innovation, other areas like disaster resistance are untouched. Permits and legal requirements are additional challenges.

Although building smart cities from ground up is a more feasible solution, it is a daunting task. The smart city concept has been experimented with several times in the past in different parts of the world. However, some have fallen victim to failure because of various reasons. Many unsuccessful attempts in the past were characterized by ambitious sizes of these cities, bold investments, technological misfits, poor urban planning, etc.


The Indian smart city landscaped must be engineered to encourage developing smaller and more realizable cities with technologies adapted to the Indian ecosystem and innovative financing. Building a strong support infrastructure, recreation options and promoting thriving businesses to flourish will make the city more habitable and desirable. Formation of communities must be allowed to follow a natural path.
Building smart cities is investment heavy and time consuming. However, India has to embrace sustainable methods soon to avoid eventual chaotic circumstances. The success of smart cities cannot be attributed to technologies alone. People must be educated about the importance and necessity of sustainable practices and the advantages of investing in sustainable settlements.