Top

Objective

Python Data Processing

I encountered a hurdle where I needed to evaluate pairs of analytical tests data for hundreds of separate comparisons. Each test produced around 130 numerical features that vary in both presence and amplitude. Manual excel comparisons was considered for individual analysis; however, proved far too cumbersome to perform hundreds of times over. To solve this problem I utilized python to automate the analysis and iterate over each comparison.

Data Cleansing SQL Data Manipulation Analytics Iteration

Author Details

Created By

Jarrad McKay


Date

September 2022

Python Scripting

Python Pandas Numpy Matplot lib PyoDBC SQL

Multivariate Comparisons

Initially the analysis looked to compare two analytical results results with one another. I needed to know how well features compare between two tests, and how well did the magnitude of the features correlate. Though comparison across the entire feature set was important, each test was composed of several different sub categories that needed to be compared in isolation as well.

Data was stored in a SQL database, and the list of comparisons was held in a basic excel file. The below code allowed for me to query appropriate data from SQL and excel, iterate over the list of comparisons, perform all the necessary calculations, and produce a dataframe that holds every calculated comparison variable across every evaluated pair inclusive of the subcategories important to each comparison.

I then wanted to evaluate how the metrics changed when features under a specific magnitude were removed from the evaluation. The below function LimitTest() allowed iteration over threshold values and the set average of each metric was captured through each threshold limit.

The below commented code reveals how these objectives were achieved.

CarbonCode




5

Countries Worked In

200M

Data Points

100K

Dollars Saved

2k

Coffee Drinked

My Projects

Turbulent Flow
Water-Soluble Drag Reducing Polymers

By: J. McKay 232 15

Some of the most widely consumed chemicals within the Oil and Gas industry are drag reducing agents. Often the industry simplifies physical characteristics of these agents to small-scale bench-top laboratory friction reduction tests. I authored this paper to tie together decades of academic research on drag reducing agents to their practical application within well completions.

This work highlights meaningful physical traits of drag reducing agents, and their practical use applications to allow operators to make better informed decisions on their chemical selections and capital allocations.

Read more
MVFF
Most Valuable Completion Fluid

By: Jarrad McKay 456 264

Clients spend millions of dollars annually stimulating wells to enhance production with the ultimate goal of getting the best return on investment. Well stimulation requires injecting fluid at high pressure and velocity into the reservoir to produce conductive fractures. This analysis was used to investigate the best return on investment when selecting a fluid composition for completions.

Read more
Python Scripting

Data Analysis with Python

By: Jarrad McKay 431 312

Deriving insights with Python. Automating multivariate comparative analysis of diagnostic results composed of samples comprised of 130 independent features for hundreds of comparisons.

Read more
JM Site

Personal Website

By: Jarrad McKay 1988 918

While learning python programming I decided to dive into web development. This site is the product of my continual learning, and drive to expand my technical knowledge.

Read more

Send a message

Get in touch

Phone :
+(720) 323 9566
Address :
Denver, Colorado
Email :
jarradmckay+inquery@gmail.com