Data

An Open Data Stack Part 3 Part 3: The Bronze Layer Previous Parts Part 1 Part 2 In this part of the series, I’m going to focus on setting just the bronze layer of my data infrastructure. I drink from the kool-aid of the medallion architecture so I’ll keep this post in line with that sort of an idea. The Bronze layer is the ‘rawest’ form of our data pipeline. It is typically an exact representation of your production data just dumped into a storage account or some commodity layer for fast, cheap, and efficient storage. This layer is optimized for reading and writing single files and is not really an exciting part of your data stack. ...

Dagster As with most initiatives in technology… It seems that my first plan is never my purest plan. Setting up Dagster in Docker was confusing for me. Honestly, I almost gave up the blog series dream as I just didn’t have a lot of time to sit down and troubleshoot. HOWEVER Here we are, to continue the good fight. I picked myself up by my shoelaces and got to work getting this thing up and running. For that reason, I’m probably going to have to break blog part 2 into a couple of parts to really get the point across. ...

An Open Data Stack But First! Looking at my poor blog, it’s been years.. YEARS since I’ve posted. A lot has changed, but a lot has stayed the same, since I last posted. Firstly, I’ve taken on a new role in my companies Architecture group. Specifically, I’ve been working on Web and Cloud applications (not new) but also our big data platform (very new). This has been a very interesting transition for me as I began my passion for computing science following my Compute 391 class where by Database instructor explained the beer and diapers correlation and I was flabbergasted by the amazing ways in which data could be utilized. ...

Man, I need to get my life figured out! I spend so much of my free time diving into new technology that I’m just waiting for my wife to pull the pin. Sorry dearest! The nerdom never dies! I just finished my application for my PMP exam (Project Management Professional - shameless self plug) and it was accepted. So once I scrounge up some dough I’ll be paying to take the test and then we are off to the races! Goal reached. ...