dateparser – python parser for human readable dates¶

dateparser provides modules to easily parse localized dates in almost any string formats commonly found on web pages.

Documentation¶

Documentation can be found here.

Features¶

Generic parsing of dates in English, Spanish, Dutch, Russian and several other langauges and formats.
Generic parsing of relative dates like: '1 min ago', '2 weeks ago', '3 months, 1 week and 1 day ago'.
Generic parsing of dates with time zones abbreviations like: 'August 14, 2015 EST', 'July 4, 2013 PST'.
Extensive test coverage.

Usage¶

The most straightforward way is to use the dateparser.parse function, that wraps around most of the functionality in the module.

dateparser.parse(date_string, date_formats=None, languages=None)[source]¶

Parse date and time from given date string.

Parameters:	date_string (str\|unicode) – A string representing date and/or time in a recognizably valid format. date_formats (list) – A list of format strings using directives as given here. The parser applies formats one by one, taking into account the detected languages. languages (list) – A list of two letters language codes.e.g. [‘en’, ‘es’]. If languages are given, it will not attempt to detect the language.
Returns:	Returns a `datetime.datetime` if successful, else returns None
Raises:	ValueError - Unknown Language

Popular Formats¶

>>> import dateparser
>>> dateparser.parse('12/12/12')
datetime.datetime(2012, 12, 12, 0, 0)
>>> dateparser.parse(u'Fri, 12 Dec 2014 10:55:50')
datetime.datetime(2014, 12, 12, 10, 55, 50)
>>> dateparser.parse(u'Martes 21 de Octubre de 2014')  # Spanish (Tuesday 21 October 2014)
datetime.datetime(2014, 10, 21, 0, 0)
>>> dateparser.parse(u'Le 11 Décembre 2014 à 09:00')  # French (11 December 2014 at 09:00)
datetime.datetime(2014, 12, 11, 9, 0)
>>> dateparser.parse(u'13 января 2015 г. в 13:34')  # Russian (13 January 2015 at 13:34)
datetime.datetime(2015, 1, 13, 13, 34)
>>> dateparser.parse(u'1 เดือนตุลาคม 2005, 1:00 AM')  # Thai (1 October 2005, 1:00 AM)
datetime.datetime(2005, 10, 1, 1, 0)

This will try to parse a date from the given string, attempting to detect the language each time.

You can specify the language(s), if known, using languages argument. In this case, given languages are used and language detection is skipped:

>>> dateparser.parse('2015, Ago 15, 1:08 pm', languages=['pt', 'es'])
datetime.datetime(2015, 8, 15, 13, 8)

If you know the possible formats that the date will be, you can use the date_formats argument:

>>> dateparser.parse(u'22 Décembre 2010', date_formats=['%d %B %Y'])
datetime.datetime(2010, 12, 22, 0, 0)

Relative Dates¶

>>> parse('1 hour ago')
datetime.datetime(2015, 5, 31, 23, 0)
>>> parse(u'Il ya 2 heures')  # French (2 hours ago)
datetime.datetime(2015, 5, 31, 22, 0)
>>> parse(u'1 anno 2 mesi')  # Italian (1 year 2 months)
datetime.datetime(2014, 4, 1, 0, 0)
>>> parse(u'yaklaşık 23 saat önce')  # Turkish (23 hours ago)
datetime.datetime(2015, 5, 31, 1, 0)
>>> parse(u'Hace una semana')  # Spanish (a week ago)
datetime.datetime(2015, 5, 25, 0, 0)
>>> parse(u'2小时前')  # Chinese (2 hours ago)
datetime.datetime(2015, 5, 31, 22, 0)

Note

Testing above code might return different values for you depending on your environment’s current date and time.

Dependencies¶

dateparser translates non-english dates to English and uses dateutil module 'parser' to parse the translated date.

Also, it requires PyYAML for its language detection module to work.

Limitations¶

Only Python 2 support for now (Python 3 support will be added in future versions)

Using DateDataParser¶

dateparser.parse() uses a default parser which tries to detect language every time it is called and is not the most efficient way while parsing dates from the same source.

dateparser.date.DateDataParser provides an alternate and efficient way to control language detection behavior.

The instance of dateparser.date.DateDataParser reduces the number of applicable languages, until only one or no language is left. It assumes the previously detected language for all the next dates and does not try to execute the language detection again after a language is discarded.

This class wraps around the core dateparser functionality, and by default assumes that all of the dates fed to it are in the same language.

class dateparser.date.DateDataParser(languages=None, allow_redetect_language=False)[source]¶

Class which handles language detection, translation and subsequent generic parsing of string representing date and/or time.

Parameters:	languages (list) – A list of two letters language codes.e.g. [‘en’, ‘es’]. If languages are given, it will not attempt to detect the language. allow_redetect_language (bool) – Enables/disables language re-detection.
Returns:	A parser instance
Raises:	ValueError - Unknown Language, TypeError - Languages argument must be a list

get_date_data(date_string, date_formats=None)[source]¶

Parse string representing date and/or time in recognizeable localized formats. Supports parsing multiple languages.

Parameters:	date_string (str\|unicode) – A string representing date and/or time in a recognizably valid format. date_formats (list) – A list of format strings using directives as given here. The parser applies formats one by one, taking into account the detected languages.
Returns:	a dict mapping keys to `datetime.datetime` object and period. For example: {‘date_obj’: datetime.datetime(2015, 6, 1, 0, 0), ‘period’: u’day’}
Raises:	ValueError - Unknown Language

Note

Period values can be a ‘day’ (default), ‘week’, ‘month’, ‘year’.

Period represent the granularity of date parsed from the given string.

In the example below, since no day information is present, the day is assumed to be current day 16 from current date (which is June 16, 2015, at the moment of writing this). Hence, the level of precision is month.

>>> DateDataParser().get_date_data(u'March 2015')
{'date_obj': datetime.datetime(2015, 3, 16, 0, 0), 'period': u'month'}

Similarly, for date strings with no day and month information present, level of precision is year and day 16 and month 6 are from current_date.

>>> DateDataParser().get_date_data(u'2014')
{'date_obj': datetime.datetime(2014, 6, 16, 0, 0), 'period': u'year'}

TODO: Timezone issues

Once initialized, dateparser.date.DateDataParser.get_date_data() parses date strings:

>>> from dateparser.date import DateDataParser
>>> ddp = DateDataParser()
>>> ddp.get_date_data(u'Martes 21 de Octubre de 2014')  # Spanish
{'date_obj': datetime.datetime(2014, 10, 21, 0, 0), 'period': u'day'}
>>> ddp.get_date_data(u'13 Septiembre, 2014')  # Spanish
{'date_obj': datetime.datetime(2014, 9, 13, 0, 0), 'period': u'day'}

Warning

It fails to parse English dates in the example below, because Spanish was detected and stored with the ddp instance:

>>> ddp.get_date_data('11 August 2012')

{‘date_obj’: None, ‘period’: ‘day’}

dateparser.date.DateDataParser can also be initialized with known languages:

>>> ddp = DateDataParser(languages=['de', 'nl'])
>>> ddp.get_date_data(u'vr jan 24, 2014 12:49')
{'date_obj': datetime.datetime(2014, 1, 24, 12, 49), 'period': u'day'}
>>> ddp.get_date_data(u'18.10.14 um 22:56 Uhr')
{'date_obj': datetime.datetime(2014, 10, 18, 22, 56), 'period': u'day'}

Documentation¶

Contents:

Installation¶

At the command line:

$ pip install dateparser

Or, if you don’t have pip installed:

$ easy_install dateparser

If you want to install from the latest sources, you can do:

$ git clone https://github.com/scrapinghub/dateparser.git
$ cd dateparser
$ python setup.py install

Deploying dateparser in a Scrapy Cloud project¶

The initial use cases for dateparser were for Scrapy projects doing web scraping that needed to parse dates from websites. These instructions show how you can deploy it in a Scrapy project running in Scrapy Cloud.

Deploying with shub¶

The most straightforward way to do that is to use the latest version of the shub command line tool.

First, install shub, if you haven’t already:

pip install shub

Then, you can choose between deploying a stable release or the latest from development.

Deploying a stable dateparser release:¶

Then, use shub to install python-dateutil (we require at least 2.3 version) and PyYAML dependencies from PyPI:

shub deploy-egg --from-pypi python-dateutil YOUR_PROJECT_ID
shub deploy-egg --from-pypi PyYAML YOUR_PROJECT_ID

Finally, deploy dateparser from PyPI:

shub deploy-egg --from-pypi dateparser YOUR_PROJECT_ID

Deploying from latest sources¶

Optionally, you can deploy it from the latest sources:

Inside the dateparser root directory:

Run the command to deploy the dependencies:

shub deploy-reqs YOUR_PROJECT_ID requirements.txt

Then, either deploy from the latest sources on GitHub:

shub deploy-egg --from-url git@github.com:scrapinghub/dateparser.git YOUR_PROJECT_ID

Or, just deploy from the local sources (useful if you have local modifications):

shub deploy-egg

Deploying the egg manually¶

In case you run into trouble with the above procedure, you can deploy the egg manually. First clone the dateparser‘s repo, then inside its directory run the command:

python setup.py bdist_egg

After that, you can upload the egg using Scrapy Cloud’s Dashboard interface under Settings > Eggs section.

Dependencies¶

Similarly, you can download source and package PyYAML and dateutil (version >= 2.3) as eggs and deploy them like above.

Contributing¶

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.

You can contribute in many ways:

Types of Contributions¶

Report Bugs¶

Report bugs at https://github.com/scrapinghub/dateparser/issues.

If you are reporting a bug, please include:

Your operating system name and version.
Any details about your local setup that might be helpful in troubleshooting.
Detailed steps to reproduce the bug.

Fix Bugs¶

Look through the GitHub issues for bugs. Anything tagged with “bug” is open to whoever wants to implement it.

Implement Features¶

Look through the GitHub issues for features. Anything tagged with “feature” is open to whoever wants to implement it.

Write Documentation¶

DateParser could always use more documentation, whether as part of the official DateParser docs, in docstrings, or even on the web in blog posts, articles, and such.

Submit Feedback¶

The best way to send feedback is to file an issue at https://github.com/scrapinghub/dateparser/issues.

If you are proposing a feature:

Explain in detail how it would work.
Keep the scope as narrow as possible, to make it easier to implement.
Remember that contributions are welcome :)

Get Started!¶

Ready to contribute? Here’s how to set up dateparser for local development.

Fork the dateparser repo on GitHub.

Clone your fork locally:

$ git clone git@github.com:your_name_here/dateparser.git

Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:
```
$ mkvirtualenv dateparser
$ cd dateparser/
$ python setup.py develop
```
Create a branch for local development:
```
$ git checkout -b name-of-your-bugfix-or-feature
```
Now you can make your changes locally.
When you’re done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:
```
$ pip install -r tests/requirements.txt # install test dependencies
$ flake8 dateparser tests
$ nosetests
$ tox
```
To get flake8 and tox, just pip install them into your virtualenv. (Note that we use max-line-length = 100 for flake8, this is configured in setup.cfg file.)

Commit your changes and push your branch to GitHub:

$ git add .
$ git commit -m "Your detailed description of your changes."
$ git push origin name-of-your-bugfix-or-feature

Submit a pull request through the GitHub website.

Pull Request Guidelines¶

Before you submit a pull request, check that it meets these guidelines:

The pull request should include tests.
If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst.
Check https://travis-ci.org/scrapinghub/dateparser/pull_requests and make sure that the tests pass for all supported Python versions.
Follow the core developers’ advices which aim to ensure code’s consistency regardless of variety approaches used by many contributors.
In case, you are unable to continue working on a PR, please leave a short comment to notify us. We will be pleased to make any changes required to get it done.

Credits¶

Committers¶

Artur Sadurski
Claudio Salazar
Elias Dorneles
Eugene Amirov
Faisal Anees
Ismael Carnales
Joseph Kahn
Mark Baas
Marko Horvatić
Mateusz Golewski
Opp Lieamsiriwong
Rajat Goyal
Raul Gallegos
Shuai Lin
Sigit Dewanto
Sviatoslav Sydorenko
Tom Russell
Umair Ashraf
Waqas Shabir

History¶

0.2.1 (2015-07-13)¶

Support for generic parsing of dates with UTC offset.
Support for Filipino dates.
Improved support for French and Spanish dates.

0.2.0 (2015-06-17)¶

Easy to use parse function
Languages definitions using YAML.
Using translation based approach for parsing non-english languages. Previously, dateutil.parserinfo was used for language definitions.
Better period extraction.
Improved tests.
Added a number of new simplifications for more comprehensive generic parsing.
Improved validation for dates.
Support for Polish, Thai and Arabic dates.
Support for pytz timezones.
Fixed building and packaging issues.

0.1.0 (2014-11-24)¶

First release on PyPI.

dateparser – python parser for human readable dates¶

Documentation¶

Features¶

Usage¶

Popular Formats¶

Relative Dates¶

Dependencies¶

Limitations¶

Using DateDataParser¶

Documentation¶

Installation¶

Deploying dateparser in a Scrapy Cloud project¶

Deploying with shub¶

Deploying a stable dateparser release:¶

Deploying from latest sources¶

Deploying the egg manually¶

Dependencies¶

Contributing¶

Types of Contributions¶

Report Bugs¶

Fix Bugs¶

Implement Features¶

Write Documentation¶

Submit Feedback¶

Get Started!¶

Pull Request Guidelines¶

Credits¶

Committers¶

History¶

0.2.1 (2015-07-13)¶

0.2.0 (2015-06-17)¶

0.1.0 (2014-11-24)¶

Indices and tables¶