What is Data Science? 

Data is all around us. From the moment we wake up and check our phones to the moment we go to sleep, data is being collected and processed to make our digital experiences more personalized and effective. Data science is the study of data, but it is also much more than that. Data scientists are not just number crunchers but also problem solvers, creative thinkers, and communicators. They use their skills to extract insights from data that can help businesses make better decisions, improve products and services, and even save lives.

Data science is a multidisciplinary field that combines mathematics, statistics, artificial intelligence, machine learning algorithms, computer engineering, and more to extract meaningful insights for people and their businesses. These insights help companies understand why or how something is happening and what is likely to happen in the future, allowing companies to prepare for and create hot trends in the market.  

‍

‍

Data Science Applications

Listing all data science applications would quickly turn this blog post into a short novel. To save us some time, data science is used for three different types of analysis: descriptive, predictive, and prescriptive.  

Descriptive Analysis

A data scientist examines data and uses a descriptive analysis to describe what is happening or what has happened in the past. Descriptive analysis is identified by data visualization graphs such as pie charts, scatter plots, histograms, etc… Descriptive analysis can help identify trends, patterns, and outliers in data and can be used to make better decisions about the future of a company.

Example:

A data scientist in a mature technology company may record the number of subscriptions sold in a year, descriptive analysis would reveal subscription spikes, slumps, and high-performing periods that can help a company understand their current state of business.  

Predictive Analysis

Predictive analysis is a data science tool that uses historical data to predict future outcomes. By analyzing patterns and trends in the data, data scientists can develop models that can predict the likelihood of certain events occurring. Predictive analytics is typically characterized by data science tools like machine learning, statistical models, and pattern matching and is used to make better decisions about the future, such as identifying risks, optimizing operations, and targeting marketing campaigns.

Example:

The same mature technology company could use predictive modeling to see which months will likely be the most (un)successful moving forward. They could use these insights for targeted advertising, deals, and pricing changes. 

Prescriptive Analysis

Prescriptive analysis combines historical data and predicted outcomes to recommend an optimal solution for forecasted events. Prescriptive analysis is typically constructed using simulations, machine learning recommendation engines, graph analysis, and more.  

Example:

A technology company may have just predicted that January is the month that sells the most subscriptions. After running some deeper prescriptive analysis, the data scientist could recommend potential reasons for January’s success and ways to make future months just as successful.

Descriptive, predictive, and prescriptive analysis all vary from one another, but are all pieces of one bigger puzzle. Skilled data scientists use all tools at their disposal to analyze data and create a digestible story for stakeholders, built out of otherwise incomprehensible sheets and numbers. 

‍

Who are Data Scientists? 

At the core, a data scientist follows a process of collecting and refining data to uncover insights that are most effective for business goals. Most data scientists work collaboratively with data engineers, data scientists, business analysts, and/or data analysts to make sure their process runs smoothly and effectively. 

‍

What is the Data Science Process? 

Data scientists are unique in their own ways, but many if not most share a pretty ‘OSEMN’ (pronounced ‘Awesome’) data science process. OSEMN is an acronym coined in 2010 by Hilary Mason and Chris Wiggins in “A Taxonomy of Data Science”, it is a five-step process that outlines the steps data scientists take to transform simple numbers into meaningful insights:

Obtain Data, Scrub Data, Explore Data, Model Data, and iNterpret results.  

‍

‍

Obtain Data

Data scientists start by gathering and collecting data from various resources. This data could be pre-existing, bought, found in a repository, simulated, etc. Each type of data source has pros and cons related to how accurate, relevant, and accessible the information is. A data scientist can choose to collect data themselves, leverage existing company data, or purchase it from third party sources (Though this can become quite expensive).

Scrub Data

One of the data science techniques for scrubbing data is called 'data mining', a process in which a scientist takes raw data and cleans it to uncover potential patterns and useful information. Once the scientist has their data collected, they must sift through and organize the unstructured data to a predetermined format. An example of scrubbing data would be writing a computer function that removes all commas from large numbers so a computer can process them as doubles or integers instead of strings. Data scrubbing sets up a data scientist to extract meaningful insights confidently and quickly from the data.

Explore Data

With a robust collection of meaningful data, scientists can begin exploring and analyzing the data. At this point, a data scientist would start with a descriptive analysis to better understand the data visualization and identify patterns and trends.

Model Data

With a better understanding of what the data contains, it is now time to apply predictive modeling and prescriptive analysis to gain more profound knowledge. Computer programs can identify which data points are associated with one another and can cluster data points based on causality. This helps data scientists understand how one component of the data affects another and how they can optimize their results.

Interpret Data

Now that we have all these models and insights for the data, it is time to work with analysts and companies to turn ideas into action. Data scientists use their created graphs and predictive models to see where a market is headed and what actions would benefit the company.

Oftentimes, this is not the end of the data science process, clients may have follow-up questions or are more interested in different parts of the market that weren’t as heavily studied. In this case, a data scientist may need to go all the way back to square one and collect more relevant data for their client’s needs or could also return to exploring and modeling data to uncover different insights.  

‍

Different Data Science Tools 

‍

Statistical Computing Tools

A data scientist must be equipped with computer science skills like data extraction, data summarization, programming skills, data visualizations, etc. They often rely on programming languages to process large amounts of data in a short time. R-studio, an open-source programming environment for statistical computing and Python, a dynamic programming language, is widely used by data scientists because it allows them to effectively analyze data. Python includes numerous libraries such as NumPy, Pandas, MATLAB, and more that are specially built to assist data scientists with data mining, data processing, data visualization tools, programming languages, programming skills, data cleansing, and more.

‍

‍

Collaborative Tools

If data scientists care to share their insights and programs, they often use GitHub, a cloud-based repository for storing, managing, and tracking all their files. Others may choose to use Jupyter Notebook, a web-based interactive computing platform that allows users to create and share code, equations, data visualization, and more. As a student who has taken Data Science classes at the University of Colorado, I personally used Python (Including NumPy and Pandas) within a Jupyter Notebook for my studies.  

‍

‍

Other Data Science Techniques for Data Analysis

On top of this, there are more broad solutions for making a data scientist’s life easier. Some leverage Artificial Intelligence and Machine Learning Models for predictive and prescriptive analysis on big data. Cloud computing gives data scientists flexible and powerful tools for processing complicated data. The Internet of Things refers to physical objects equipped with sensors and software that can connect to the internet. These devices collect and generate massive amounts of data that scientists can extract. Finally, some data scientists use quantum computers that perform complex calculations in record time. Some may go as far as building complex quantitative algorithms made to perform extremely large and complex functions for data analytics.  

‍

Methods for Storing Data and Effective Data Management

Data storage is a critical aspect of managing and analyzing vast amounts of data. Various methods and technologies are available to store and access data efficiently. In this section, we will explore different data storage methods and their applications.

Data Warehouses 

A data warehouse is a centralized storage system that contains huge amounts of current and historical data. Data flows into a data warehouse from transactional systems and other databases, this in turn creates a huge central repository for scientists to analyze and query data to gain meaningful insights.  

Database 

A database is a collection of structured data typically stored in a computer system and managed by a database management system (DBMS). The difference between a database and a data warehouse is that a database is a collection of data typically stored for a specific business application that is organized in a way that it is easy to access and manipulate. A data warehouse is a large-scale central repository that collects various data from multiple databases for the purpose of analysis and reporting.

Cloud Computing 

Cloud computing is the delivery of IT resources, including storage, databases, software, and analytics in the cloud (through the internet). Cloud computing is useful because it uses remote servers to store and access data at scale. Many companies team up with Dropbox and Google because they can bring traditional files and cloud content together in one place. This allows for a central, collaborative place for data scientists to host files.

Computer Memory 

Computer memory stores data and programs for immediate use within the computer. You may have heard of random access memory (RAM), which is responsible for ‘short term memory’ and enabling your computer processors to read data to run applications and open files.

File System 

A file system is responsible for naming, storing, and retrieving files within your computer. Without a file system, your computer files would be completely unstructured and very difficult to retrieve. 

Object Storage

Object storage is a technology that stores data as ‘objects’ as opposed to file systems or block storage. Companies use object storage to create and analyze large volumes of unstructured data such as videos, images, emails, etc. using a unique identifier for quick access and retrieval.

‍

Hybrid Cloud storage 

Hybrid Cloud solutions take the best of both worlds from both cloud and on-premises resources. It combines the functionalities of both public and private clouds to form a robust architecture. Hybrid cloud solutions are oftentimes very flexible, scalable, and cost-effective. 

Efficient data storage methods are essential for managing and analyzing large volumes of data. Data warehouses, databases, cloud computing, collaborative file hosting, object storage, computer memory, file systems, and hybrid cloud storage offer diverse options for effective data management. Understanding these storage methods allows organizations and data scientists to optimize their data infrastructure and derive valuable insights from their data assets.

‍

‍

Challenges in Data Science 

‍

Multiple data pipelines

As seen above, there are tons of different ways to store and format data. As a result, data scientists often find themselves with different types of apps and tools that generate multiple data sources in various formats. Having to clean, prepare, and get all data points in the same format is oftentimes very tedious and time consuming. Though programming languages help with this, it is still very difficult to manage data coming in from all different directions. 

Miscommunication

Another common obstacle for data scientists is the disconnect between what your business wants done, and what can realistically be accomplished. For those involved in business operations, many people struggle understanding the technology and processes for data scientists. On the other hand, many data scientists struggle to understand the goals and vision of the company and tend to bury their heads in numbers. It is very important to maintain transparency and have specific, tangible goals when communicating with multiple managers that have various requirements. Data Science is used to uncover meaningful insights about a business, but if there isn't transparency, data science team members struggle to extract data that is actually relevant to the corporation.

Too much Data

A big challenge in the past has been needing data, and not having it. As this problem grew, more companies looked to solve the data acquisition issue by developing more IoT devices and creating data warehouses for a massive, central source of data. Now, gathering data isn’t an issue, a problem now arises in having to find that needle of information in a haystack of numbers. Data scientists are still looking for solutions to this problem, leveraging artificial intelligence and machine learning to quickly and accurately work with scalable data through all parts of the ‘OSEMN’ process.  

Machine Learning Bias

Another problem data scientists often deal with is obtaining inaccurate outcomes as a result of machine learning bias. For example, if an algorithm is trained based on data from mature companies, it may produce less accurate results given a list of startups to process. Because of this, data scientists must constantly be looking for where bias may exist and to what extent it affects the outcome. This may seem easy at first, but bias can exist all the way down to the 1’s and 0’s of the computer.  

Data Science compared to other fields

‍

Difference between Data Science and Data Analytics

The difference between data science and data analytics lies in their scope and focus. Data analytics is a subset of data science, with data science encompassing all aspects of data processing, from collection to modeling to insights. A data analyst primarily revolves around statistics, mathematics, data analysis, and statistical analysis, whereas data science professionals deal with the broader organizational context of data.

Data analysts typically dedicate more time to routine data analysis and generating regular reports. They make sense out of existing data, providing valuable insights. On the other hand, a data scientist is involved in designing the way data is stored, manipulated, and analyzed. They create new methods and tools to process data that can be utilized by analysts.

Difference between Data Science and Business Analytics

When comparing data science and business analytics, data science professionals tend to work more closely with data technology, while business analysts bridge the gap between business and IT. Business analysts define business cases, gather information from stakeholders, and validate solutions. Data scientists, on the other hand, employ technology to work with business data, writing programs, applying machine learning techniques to create models, and developing new algorithms. The output from data scientists is then utilized by a business analyst to communicate a coherent story that the broader business can understand.

Difference between Data Science and Statistics

Data science and statistics differ in their nature and objectives. Statistics is a mathematically-based field that aims to collect and interpret quantitative data. It focuses on analyzing data using statistical methods to draw conclusions and make inferences. On the other hand, data science is a multidisciplinary field that employs scientific methods, processes, and systems to extract knowledge from data in various forms. It encompasses statistics as one of its components but also incorporates other disciplines such as computer science, machine learning, and domain expertise to derive insights from data.

‍

Why become a DataScientist?

‍

Why is data science such a hot career choice? There are a few reasons. First, the amount of data that is being generated is exploding. This means there is a growing need for people who can analyze and interpret this data. Second, data science is a versatile field that can be applied to various industries. This means that data scientists have many job opportunities to choose from.

Steps you can take to prepare for a career in data science:

1. Get a strong foundation in mathematics, statistics, and computer science. This will give you the basic knowledge to understand and analyze data. You can take courses in these subjects at a university or community college or learn them online.
2. Gain experience with data analysis and machine learning tools. Many different tools are available for data science, so it's important to get some hands-on experience with them. You can do this by taking online courses, participating in hackathons, or working on personal projects.
3. Network with other data scientists. This is a great way to learn about the latest data science trends and get your foot in the door. You can attend meetups, conferences, and online forums to meet other data scientists.
4. Get a degree in data science. A degree in data science can give you the skills and knowledge you need to be a successful data scientist. However, having a degree to get a job in data science is not essential. Many data scientists have learned their skills on the job or through online courses.

Data science is a rewarding career that offers the opportunity to make a real impact on the world. If you are interested in a challenging and exciting career with a bright future, then data science is the perfect choice.