Question
Translate Pandas groupby plus resample to Polars in Python
I have this code that generates a toy DataFrame (production df is much complex):
import polars as pl
import numpy as np
import pandas as pd
def create_timeseries_df(num_rows):
date_rng = pd.date_range(start='1/1/2020', end='1/01/2021', freq='T')
data = {
'date': np.random.choice(date_rng, num_rows),
'category': np.random.choice(['A', 'B', 'C', 'D'], num_rows),
'subcategory': np.random.choice(['X', 'Y', 'Z'], num_rows),
'value': np.random.rand(num_rows) * 100
}
df = pd.DataFrame(data)
df = df.sort_values('date')
df.set_index('date', inplace=True, drop=False)
df.index = pd.to_datetime(df.index)
return df
num_rows = 1000000 # for example
df = create_timeseries_df(num_rows)
Then perform this transformations with Pandas.
df_pd = df.copy()
df_pd = df_pd.groupby(['category', 'subcategory'])
df_pd = df_pd.resample('W-MON')
df_pd.agg({
'value': ['sum', 'mean', 'max', 'min']
}).reset_index()
But, obviously it is quite slow with Pandas (at least in production). Thus, I'd like to use Polars to speed up time. This is what I have so far:
#Convert to Polars DataFrame
df_pl = pl.from_pandas(df)
#Groupby, resample and aggregate
df_pl = df_pl.group_by(['category', 'subcategory'])
df_pl = df_pl.group_by_dynamic('date', every='1w', closed='right')
df_pl.agg([
pl.col('value').sum().alias('value_sum'),
pl.col('value').mean().alias('value_mean'),
pl.col('value').max().alias('value_max'),
pl.col('value').min().alias('value_min')
])
But I get AttributeError: 'GroupBy' object has no attribute 'group_by_dynamic'
. Any ideas on how to use groupby
followed by resample
in Polars?
7 136
7