Dean Marchiori - Blog

Integrating R with Modern Tech Stacks

A Practical Guide to Using R and Docker for Integrating Statistical Analysis into Modern Tech Stacks

Docker

R

There seems to be a subconscious deterrent from using specialised tools like R as part of a production or core tech stack, as if it’s somehow not compatible with other forms…

5 tips for dealing with IT

Getting data science done with your IT team

Analysis

R

No plan survives first contact with IT

OceaniaR 2024

R Development and Open Source Software Hackathon - Melbourne 2024

Analysis

R

The first ever OceaniaR meeting took place this week in Melbourne!

Data Systems and Analytics for Vector-Borne Disease Threat Reduction

A real-world case study of reproducible workflows

Analysis

R

Earlier this year I was fortunate enough to work with EcoHealth Alliance on a major project focused on two WHO priority zoonoses, Rift Valley fever virus and Crimean-Congo…

Data Science Workflows: Choosing the Right One

Do a Repro-Retro

Analysis

In previous posts I have discussed frameworks for thinking about data science projects. When it comes to actually writing R code, there are a number of coding workflows you…

Data Science Workflows: Inner Loop vs Outer Loop

Experiment and Deploy

Analysis

In a previous post I discussed the charming nostalgia of CRISP-DM as a data analysis workflow choice.

Data Science Workflows: CRISP-DM

90’s nostaglia that holds up

Analysis

The cross-industry standard process for data mining (CRISP-DM) is a process model and framework for carrying out data mining projects.

Is ChatGPT is a bullshit machine?

And what you should do about it

AI

A recent paper published in ‘Ethics and Information Technology’ titled ChatGPT is bullshit ¹ is fast becoming one of my favorite papers. Not least because it uses the word…

Is the share market just random noise?

Modelling market prices using a random walk

R

Markets

Earlier in my career, in the afternath of the GFC I worked as trader for an online share trading platform. Everyday I would get blasted with a firehose of emotion from…

Geolocating Sydney’s weirdest property

Using Open Street Map and R to geolocate an image

R

OSM

A defiant Aussie family has refused to sell their farm-land property despite the entire neighborhood being converted into a new housing estate.

Deploying Enterprise Scale AI & Machine Learning Infrastucture

5 things you need to know before using Azure ML Studio

analysis

statistics

Are you about to scale up your data analytics team to do more AI/ML work? Here are 5 things you need to know up front to make your life easier.

Three Questions to Ask Your Data Scientist

Are your predictive models doing what you think they are?

analysis

statistics

If you have hired a data scientist or run a team of data people, you may not be an expert yourself (that’s why you hired an expert, right?). So how do you know you are…

Why your data science projects are failing

and what you can’t do about it

analysis

The most undervalued skill in delivering value with data science teams is picking projects that are likely to succeed. There is no shortcut - it takes years of hard earned…

Man vs Machine Learning

I went head-to-head with Microsoft’s AutoML platform in a predictive modelling challenge.

R

Azure

Over the past few years I have been doing more and more work in Microsoft Azure and the Azure Machine Learning Studio. One feature of the Azure ML studio is an automated…

Level up your Forecasting

Quantify uncertainty with distributional forecasts

R

statistics

This is a time series data set of half-hourly electricity demand for Victoria, Australia ¹. You may not care about electricity forecasting, but there is probably some…

Random Number Generator Testing

A tale of an unemployed ice-cream man, secret societies and a $10 radio

R

statistics

We had really interesting discussion with a company recently who needed a random number generator certified.

Deploy Your Own R Data Science Lab in the Cloud

R

azure

In a previous post I linked to a project that makes it easy to deploy and extend an existing Rocker Project Docker image to quickly set up a fully featured RStudio Server…

Building your own R Data Science Lab in the browser

R

docker

Most R users probably just run RStudio Desktop from Posit on their local computers. This involves manually installing R, RStudio and all other packages.

Beware of Boundaries in Binominal Proportion Confidence Intervals

R

statistics

Binomial proportion confidence intervals are often employed when attempting to perform tests for significance, or sample size calculations around sample measurements…

When should you be using the Hypergeometric distribution in practice?

R

statistics

We have a manufacturing process in the day job that is subject to sample auditing.

Prediction Intervals for Linear Mixed Effects Models

R

statistics

A recent project with repeated measures data involved fitting a random intercept term, and eventually making predictions for new groups not in the training sample.…

Optimal Stopping Problems

R

statistics

The August 2022 edition of Significance Magazine posts a challenge:

Three Stick Statistics

statistics

When I was a teenager I was very popular among my peers, which is why I spent every second Saturday playing in a retiree’s social golf club. For each event we would travel…

What was that Bluey Episode?

R

shiny

Does your child explain Bluey episodes to you but you have no idea what episode they are talking about and can’t handle flicking through all 131 of them?

Running Shiny in a Docker container

R

analysis

Basic minimal example for running shiny in Docker. It is assumed you have Docker installed.

Mapping NSW Fire Incidents in R

R

This feed contains a list of current incidents from the NSW RFS, and includes location data and Major Fire Update summary information where available. Click through from the…