By Dr. Deanne Larson
Faculty Member, Information Technology at American Public University
The consistent evolution of technology has produced the need for new IT skills over the last two decades. Couple this with the explosion of data, and it shouldn’t be surprising to see a demand for IT resources that support business analytics and the processing of what is known as Big Data.
Big Data categorizes extremely large data sets produced by website traffic, social media, machine-to-machine communications, equipment sensors, and documents; this is not an exhaustive list; but you get the picture. Why is this data any different from data captured by traditional organizational information systems? The volume, variety, and velocity of the data distinguishes Big Data from other data sets. Unstructured data, level of detail, and overall amount generated challenges what we know about data management.
Why is Big Data important and why should you care? New careers are being created because of Big Data. An increased demand for data scientists, business analytics specialists, and technologists who can prepare process, analyze, and find value in Big Data are highly sought after. Many companies such as Amazon, Google, Ford, and Capital One build organizational strategy around analytics. Why? Analytics and Big Data provide organizations with a competitive edge. In place of chasing competition, these companies find opportunities to change industries by offering innovative products and services based on results from the analysis of Big Data.
Big Data careers are still evolving, but at the moment the demand is for individuals who can apply statistics to large data sets and those that can prepare and process Big Data. While the job title of data scientist might be intimidating, this role is evolving and very broad. Data scientist job descriptions often require a higher degree in statistics, the ability to use analytical software such as R (an open source statistical programming language) or SAS, or code in Python or other object-oriented languages. Big Data technologist job descriptions include using new technologies such as Hadoop (a Big Data file system for storing data), Java, and new languages such as Pig (yes, this is the real name) and Hive to manipulate and analyze data. The results of these roles working together often produce new discoveries for organizations such as which products are purchased together or gaining a deeper understanding of customer behavior.
The demand for Big Data resources is clear; all you need to do is search on “Big Data” or “data scientist” on LinkedIn and other job boards to see the demand. A shortage exists and experienced individuals are in short supply, so companies are looking for the right people with the right skills—making this an interesting career path to consider. Science, Technology, Engineering, and Math (STEM) degree programs align well with careers in Big Data. If you are looking for a challenging and visible position, Big Data might be for you.
About the Author
Dr. Deanne Larson is an Associate Professor at American Public University System (APUS) in the School of Science, Technology, Engineering, and Math. Dr. Larson consults and researches in the areas of Data Warehousing, Business Intelligence, and Big Data.