Fleming - Making Datetime Manipulation in Python Easy
Ambition Co-Founder and CTO Wes Kendall explains how Ambition set up games in Ambition to work across different time zones, thus allowing an office in Singapore to compete with an office in Cleveland, Ohio.
We just open-sourced Fleming, a python library for working with datetimes across timezones. Hope you enjoy!
We do a lot of datetime manipulation in Python here at Ambition, and we do it all with respect to our users’ timezones. Chunking time into larger units, aggregating team metrics over members that are geographically separate, and scheduling competitions and triggers are a few examples of some of the things we have found challenging to solve.
Manipulating datetime objects with respect to timezones in Python can be tricky. Luckily, pytz solves the basic cases of doing conversion from one timezone into another. However, some common pitfalls still occur when doing other types of datetime manipulation, such as datetime arithmetic across Daylight Savings Time boundaries. Want an example? Here is one straight from Ambition.
Scheduling Weekly Games in Ambition - Our Datetime Woes
If you don’t already know, Ambition schedules weekly games for teams in a football-like schedule. The very first season of our first customer started on October 7th and ended December 20th. They configured their games to start at midnight on Monday and end at 5 PM on Friday every week in Eastern Standard Time (EST).
Our first naive version of code was structured to schedule their games like the following:
# Get a UTC season start time for midnight Monday EST. season_start = datetime(2013, 10, 7, 4) # Schedule a 10-week season for week_num in range(10): # Every game starts at midnight on Monday EST game_start = season_start + timedelta(weeks=week_num) # Make the game end on Friday at 5PM EST game_end_ = game_start + timedelta(days=4, hours=17) # Create the game Game.objects.create(...)
The game start and end times for each week then looked like this:
Week 1: 2013-10-07 04:00:00 - 2013-10-11 21:00:00 Week 2: 2013-10-14 04:00:00 - 2013-10-18 21:00:00 Week 3: 2013-10-21 04:00:00 - 2013-10-25 21:00:00 Week 4: 2013-10-28 04:00:00 - 2013-11-01 21:00:00 Week 5: 2013-11-04 04:00:00 - 2013-11-08 21:00:00 Week 6: 2013-11-11 04:00:00 - 2013-11-15 21:00:00 Week 7: 2013-11-18 04:00:00 - 2013-11-22 21:00:00 Week 8: 2013-11-25 04:00:00 - 2013-11-29 21:00:00 Week 9: 2013-12-02 04:00:00 - 2013-12-06 21:00:00 Week 10: 2013-12-09 04:00:00 - 2013-12-13 21:00:00
Programmers that have dealt with datetime arithmetic in Python might already be doing a face palm, but even experienced programmers sometimes don’t recognize the bug that just happened.
A while after we scheduled our first season for our first customer, we noticed that games were starting and ending an hour earlier than they used to. We then realized that Daylight Savings Time (DST) stopped on November 3rd, right in the middle of the season. If you convert the dates after November 3rd above into EST, you’ll notice that they are all an hour earlier than originally intended.
It was obvious to us that our first approach of doing datetime arithmetic in UTC time was somewhat foolish, but you live and you learn. We opted to do the arithmetic in the timezone of the season instead (i.e. using “aware” datetime objects that had timezone information rather than “naive” ones).
The new code looked something like this:
est = pytz.timezone('US/Eastern') # Get a local season start time for midnight Monday EST. season_start = est.normalize( datetime(2013, 10, 7, 4, tzinfo=pytz.utc)) # Schedule a 10-week season for week_num in range(10): # Every game starts at midnight on Monday EST game_start = season_start + timedelta(weeks=week_num) # Make the game end on Friday at 5PM EST game_end = game_start + timedelta(days=4, hours=17) # Call normalize to account for DST transitions game_start = est.normalize(game_start) game_end = est.normalize(game_end) # Create the game Game.objects.create(...)
When using this code on the values, we end up with the following results:
Week 1: 2013-10-07 00:00:00-04:00 - 2013-10-11 17:00:00-04:00 Week 2: 2013-10-14 00:00:00-04:00 - 2013-10-18 17:00:00-04:00 Week 3: 2013-10-21 00:00:00-04:00 - 2013-10-25 17:00:00-04:00 Week 4: 2013-10-28 00:00:00-04:00 - 2013-11-01 17:00:00-04:00 Week 5: 2013-11-03 23:00:00-05:00 - 2013-11-08 16:00:00-05:00 Week 6: 2013-11-10 23:00:00-05:00 - 2013-11-15 16:00:00-05:00 Week 7: 2013-11-17 23:00:00-05:00 - 2013-11-22 16:00:00-05:00 Week 8: 2013-11-24 23:00:00-05:00 - 2013-11-29 16:00:00-05:00 Week 9: 2013-12-01 23:00:00-05:00 - 2013-12-06 16:00:00-05:00 Week 10: 2013-12-08 23:00:00-05:00 - 2013-12-13 16:00:00-05:00
As you can notice from the values, the timezones change properly when transitioning out of DST (i.e. going from -04:00 to -05:00); however, the time values themselves are still one hour behind after the transition.
Solving the Datetime Crisis with Fleming, our Open-Source Datetime Library
Pytz and the regular Python timedelta objects are not responsible for our woes. In fact, they are operating exactly how they are documented to operate. Ambition simply needed it to work in a way that made more sense for our users. For example, even though 25 hours had passed on November 23rd for our EST users, we still wanted our datetime arithmetic to operate like 24 hours had passed.
To handle datetime arithmetic (and other datetime manipulation functions), we implemented Fleming. Fleming is an open source python library freely available here. Installing Fleming is as simple as
pip install fleming
Fleming allows you to work with datetimes that are naive or aware and also allows you to work with datetimes with respect to other timezones. For example, consider our previous problem of scheduling. Fleming provides an “intervals” function that allows the user to give a start time (a datetime object), an interval (a timedelta object), and a count of intervals. It returns a generator of datetime objects starting at start_time and going for count intervals.
import fleming from datetime import datetime, timedelta intervals = fleming.intervals( datetime(2013, 10, 7, 4), timedelta(weeks=1), count=10) for i in intervals: print i 2013-10-07 04:00:00+00:00 2013-10-14 04:00:00+00:00 2013-10-21 04:00:00+00:00 2013-10-28 04:00:00+00:00 2013-11-04 04:00:00+00:00 2013-11-11 04:00:00+00:00 2013-11-18 04:00:00+00:00 2013-11-25 04:00:00+00:00 2013-12-02 04:00:00+00:00 2013-12-09 04:00:00+00:00
What’s that you say? It didn’t handle the DST transition? That’s because we used a naive UTC time as our start time. Let’s try it out with an aware EST start time.
import pytz est_start = pytz.timezone('US/Eastern').normalize( datetime(2013, 10, 7, 4), tzinfo=pytz.utc) intervals = fleming.intervals( est_start, timedelta(weeks=1), count=10) for i in intervals: print i 2013-10-07 00:00:00-04:00 2013-10-14 00:00:00-04:00 2013-10-21 00:00:00-04:00 2013-10-28 00:00:00-04:00 2013-11-04 00:00:00-05:00 2013-11-11 00:00:00-05:00 2013-11-18 00:00:00-05:00 2013-11-25 00:00:00-05:00 2013-12-02 00:00:00-05:00 2013-12-09 00:00:00-05:00
And voila, even though DST stopped, the times still remained at midnight in EST.
The intervals function also allows you to provide a naive UTC start time and do intervals with respect to another time zone while returning the times in UTC. This is accomplished by using the within_tz parameter.
est = pytz.timezone('US/Eastern') intervals = fleming.intervals( datetime(2013, 10, 7, 4), timedelta(weeks=1), count=10, within_tz=est) for i in intervals: print i 2013-10-07 04:00:00+00:00 2013-10-14 04:00:00+00:00 2013-10-21 04:00:00+00:00 2013-10-28 04:00:00+00:00 2013-11-04 05:00:00+00:00 2013-11-11 05:00:00+00:00 2013-11-18 05:00:00+00:00 2013-11-25 05:00:00+00:00 2013-12-02 05:00:00+00:00 2013-12-09 05:00:00+00:00
See how the hour goes from 4 to 5 after the DST transition? This allows you to still work in UTC while doing calculations with respect to other timezones - an added win for simplicity in Ambition's codebase.
Other Capabilities of Fleming
We have also provided other capabilities in Fleming that helped solve many of our common datetime manipulation problems. One example is datetime truncation. Do you need to round a datetime down to its nearest month, day, week, year, hour, minute, or second? Use the floor function:
import fleming from datetime import datetime print fleming.floor(datetime(2013, 10, 3), month=1) 2013-10-01 00:00:00+00:00
Do you want to round a UTC time down with respect to another timezone and still return it in UTC? Use the within_tz argument. In our previous example, it was the 3rd of October midnight UTC, however, it is still October 2nd in EST, so rounding down a day produces this:
import pytz print fleming.floor( datetime(2013, 10, 3), day=1, within_tz=pytz.timezone('US/Eastern')) 2013-10-02 00:00:00+00:00
What if your time is originally out of DST but goes in DST because of the floor? Fleming still produces a time with the right timezone and time values:
est_time = pytz.timezone('US/Eastern').normalize( datetime(2013, 11, 28, tzinfo=pytz.utc)) print est_time 2013-11-27 19:00:00-05:00 print fleming.floor(est_time, month=1) 2013-11-01 00:00:00-04:00
We also provide the ability to do the ceil of a time, get unix times with respect to timezones, convert naive and aware datetimes to other timezones with ease, and allow you to do arbitrary datetime arithmetic. For complete descriptions and usage examples of our Fleming library, go here.
This is our first full open source project release at Ambition, and we are curious how to make it better. Feel free to leave any comments or suggestions below. We hope that we have helped solve your datetime woes as well!
Wes Kendall is the Co-Founder and CTO of Ambition. He runs the Ambition Engineering Team and is the driving force behind our industry leading sales productivity and analytics platform.
Learn More About Ambition
Ambition’s acclaimed employee productivity platform gives 360° visibility into individual and team performance.
Create accountability and recognition with live performance data from any data source. Track and broadcast key metrics to personalized dashboards and office TVs. Put holistic goals right in front of your reps. Compare activity level and goal attainment across teams to see how hard and how smart your reps are working. Benchmark success for teams, roles and individuals, then drive results via automated scorecards, contests, recognition and reporting.
Perfect for front office teams that value performance-driven culture and transparent operations, Ambition is a Harvard Business Review and AA-ISP endorsed solution for driving frontline revenue. See how clients like PwC, Lyft, Wayfair, FiveStars, Total Quality Logistics, and Outreach use Ambition at ambition.com.