How can Big Data solutions transform the oil and gas industry?

Apr 01,2019 by Parul Singh

Data has been equated with oil on more than one occasion. Ironically, oil and gas is one of those few industries which are yet to witness large-scale deployment of Big Data solutions.

Most of the oil and gas companies still use analytical solutions on a small-scale, for instance, for monitoring and comparing equipment in a single off-shore refinery. Oil and gas industry has always been averse to the adoption of new technologies and Big Data comes as no exception.

Oil and Gas: An industry rife with technical complexities and risks

Oil and gas industry has always been associated with astronomical revenues. The harsh truth is that oil and gas companies do not achieve a profit margin of more than 8-9%.

Several external factors affect the profitability of this industry.

Oil and gas prices are highly volatile and susceptible to the political and economic stability of a region.

Of late, environmentally conscious governments have begun to emphasize the use of renewable energy. Such alternative forms of energy can diminish the demand for oil and gas. This can affect the bottom-line of any company.

Oil and gas has always been a technically complex and economically risky space.

The sheer amount of technological expertise, manpower and machinery involved makes oil an expensive proposition.

Discovering, drilling and extracting oil entails a lot of capital expenditure.

Oil is difficult to find. Contrary to popular belief, oil is not found in pools of rocks. It is rather found trapped in tiny pores between layers of rocks.

Here you can picture a sponge holding water. Rocks hold oil between their pores just as a sponge holds water in its pores.

Oil is typically extracted from rocks that have pores large enough to let the fluid flow through. Extracting oil from smaller and more complex pores is difficult and economically unviable.

A considerable part of the world’s hydrocarbons are locked in deep waters or terrains difficult to drill through.

See also  Big Data for the Travel Industry: All You Need to Know!

The reservoirs containing oil or gas are typically found 5,000 to 35,000 feet below the Earth’s surface.

The rocks that are drilled are complex for the fluids to move through. Then, the fluids themselves have several different chemical and physical properties that can make extraction difficult.

Extraction of oil and gas also comes with an array of environmental and human safety concerns that need to be addressed.

Big Data can be a disrupter for Oil and Gas industry

Oil and gas companies typically spend a lot of time in exploring oil. Geologists and petro physicists identify the rocks in nearby wells using the data available with them.

This data may have high resolution but usually provides information to a small range i.e. up to 10 feet around the well. To make up for this inadequacy, scientists have to apply a significant amount of interpretation to the data to derive meaningful, actionable information.

Geologists use seismic surveys to explore oil in both onshore and offshore locations.

In a seismic survey, a shock wave (called seismic wave) is created on the surface of the earth using a source of energy. This shock wave travels down into the earth, gets reflected by sub-surface formations and returns to the surface where it is captured by devices (called receivers). The data thus recorded is processed to derive meaningful information.

Data collected through seismic surveys has low resolution. Scientists, therefore, combine this data with well log data from nearby oil wells to create a better picture of the sub-surface.

A well log provides information on the rocks that were cut and the fluid that was extracted during drilling.

This data, however, has its limitations.

Well logs from different wells have different formats and scales. Moreover, these datasets are humongous. Therefore, there is a dire need to bring in Big Data solutions that can integrate data from several different sources to produce meaningful insights.

See also  Big Data Empowers Media & Entertainment

Figure 2: Oil companies can use predictive analytics to minimize the uncertainty involved in exploration and drilling.

Oil drilling and extraction operations are unusually expensive. A single offshore well costs up to millions of dollars and this figure increases as the depth of the reservoir increases, requiring more complex technology.

The average cost of exploration and development for a barrel of oil ranges between 7 and 15 dollars. It, therefore, makes sense to proceed with the extraction of oil only if a reservoir has enough oil. If oil is present in small pockets, companies find it difficult to control development costs and maintain profit margins.

Oil companies need Big Data-like systems that can minimize the trial and error involved in exploration by accurately predicting information about any reservoir i.e. amount of oil present, the texture of the rock, etc.

For this purpose, companies can use data collected during drilling operations.

Drilling involves the use of extensive machinery, measurement devices and manpower, all of which capture data in different forms: videos, images and structured data.

Hadoop-like solutions can be employed to carry out predictive analytics using this data. Predictive analytics can minimize the uncertainty factor involved in exploring and drilling oil.

Now the question is how can Big Data solutions possibly help when the drilling environments vary so widely?

The answer lies in the fact that grains of similar rocks behave similarly everywhere. So, the grains may not be laid down the exact same way in any two sites, but the lessons learned in one site can be extrapolated to the other.

Big Data solutions can integrate all the information available across thousands of reservoirs across the globe. If oil is to be drilled in a new site, this data can be used to find the best possible method to drill it. Predictive analytics can take into account the local conditions and identify how to proceed.

See also  Top 10 Big Data Analytics Use Cases and Applications for Global Industries

Companies can install IoT sensors in equipment and track the drilling process in real-time to minimize or altogether prevent any potential danger.

These sensors track all elements that impact products such as wave heights, temperature, humidity, etc.

These sensors also monitor the equipment used in extraction to prevent any unplanned downtime. Companies can schedule preventative maintenance for all equipment and prolong their shelf-life. Such a pro-active approach can help companies reduce the cost of operations significantly.

Petronas, one of the leading oil and gas major from Malaysia, has taken a big leap in this direction. The company has partnered with Aspen Technology, an asset optimization software company to optimize operations at its facility in Pengerang, Johor.

Big Data solutions can also play a crucial role when it comes to audits. Safety, compliance and audit checks are common to any oil and gas company.

If audits are done digitally, anyone in the company can monitor an audit in real-time and check its impact to find if it was done correctly.

They can answer questions such as “How many times a particular filter was changed” or “When was the last health check-up done” in a jiffy.

Companies can also collate data from day-to-day operations and figure out ways to reduce costs, wherever possible.

There are real-life examples of solutions like GDS Ware that are helping oil and gas companies optimize their daily operations. Companies can gather real-time data across a team of professionals that can be instantly used for decision making.


With so many use-cases, Big Data sure comes as a disruptive factor in the oil and gas industry. Companies need to tackle inertia, lack of technical-know-how and bring in appropriate changes at technology and manpower level to harness its potential to the maximum.

Leave a Reply

Notify of