This is a major release from 0.17.1 and includes a small number of API changes, several new features, enhancements, and performance improvements along with a large number of bug fixes. We recommend that all users upgrade to this version.
Warning
pandas >= 0.18.0 no longer supports compatibility with Python version 2.6 and 3.3 (GH7718, GH11273)
numexpr version 2.4.4 will now show a warning and not be used as a computation back-end for pandas because of some buggy behavior. This does not affect other versions (>= 2.1 and >= 2.4.6). (GH12489)
numexpr
Highlights include:
Moving and expanding window functions are now methods on Series and DataFrame, similar to .groupby, see here.
.groupby
Adding support for a RangeIndex as a specialized form of the Int64Index for memory savings, see here.
RangeIndex
Int64Index
API breaking change to the .resample method to make it more .groupby like, see here.
.resample
Removal of support for positional indexing with floats, which was deprecated since 0.14.0. This will now raise a TypeError, see here.
TypeError
The .to_xarray() function has been added for compatibility with the xarray package, see here.
.to_xarray()
The read_sas function has been enhanced to read sas7bdat files, see here.
read_sas
sas7bdat
Addition of the .str.extractall() method, and API changes to the .str.extract() method and .str.cat() method.
pd.test() top-level nose test runner is available (GH4327).
pd.test()
Check the API Changes and deprecations before updating.
What’s new in v0.18.0
New features
Window functions are now methods
Changes to rename
Range index
Changes to str.extract
Addition of str.extractall
Changes to str.cat
Datetimelike rounding
Formatting of integers in FloatIndex
Changes to dtype assignment behaviors
to_xarray
Latex representation
pd.read_sas() changes
pd.read_sas()
Other enhancements
Backwards incompatible API changes
NaT and Timedelta operations
Changes to msgpack
Signature change for .rank
Bug in QuarterBegin with n=0
Resample API
Downsampling
Upsampling
Previous API will work but with deprecations
Changes to eval
Other API changes
Deprecations
Removal of deprecated float indexers
Removal of prior version deprecations/changes
Performance improvements
Bug Fixes
Contributors
Window functions have been refactored to be methods on Series/DataFrame objects, rather than top-level functions, which are now deprecated. This allows these window-type functions, to have a similar API to that of .groupby. See the full documentation here (GH11603, GH12373)
Series/DataFrame
In [1]: np.random.seed(1234) In [2]: df = pd.DataFrame({'A': range(10), 'B': np.random.randn(10)}) In [3]: df Out[3]: A B 0 0 0.471435 1 1 -1.190976 2 2 1.432707 3 3 -0.312652 4 4 -0.720589 5 5 0.887163 6 6 0.859588 7 7 -0.636524 8 8 0.015696 9 9 -2.242685 [10 rows x 2 columns]
Previous behavior:
In [8]: pd.rolling_mean(df, window=3) FutureWarning: pd.rolling_mean is deprecated for DataFrame and will be removed in a future version, replace with DataFrame.rolling(window=3,center=False).mean() Out[8]: A B 0 NaN NaN 1 NaN NaN 2 1 0.237722 3 2 -0.023640 4 3 0.133155 5 4 -0.048693 6 5 0.342054 7 6 0.370076 8 7 0.079587 9 8 -0.954504
New behavior:
In [4]: r = df.rolling(window=3)
These show a descriptive repr
In [5]: r Out[5]: Rolling [window=3,center=False,axis=0]
with tab-completion of available methods and properties.
In [9]: r.<TAB> # noqa E225, E999 r.A r.agg r.apply r.count r.exclusions r.max r.median r.name r.skew r.sum r.B r.aggregate r.corr r.cov r.kurt r.mean r.min r.quantile r.std r.var
The methods operate on the Rolling object itself
Rolling
In [6]: r.mean() Out[6]: A B 0 NaN NaN 1 NaN NaN 2 1.0 0.237722 3 2.0 -0.023640 4 3.0 0.133155 5 4.0 -0.048693 6 5.0 0.342054 7 6.0 0.370076 8 7.0 0.079587 9 8.0 -0.954504 [10 rows x 2 columns]
They provide getitem accessors
In [7]: r['A'].mean() Out[7]: 0 NaN 1 NaN 2 1.0 3 2.0 4 3.0 5 4.0 6 5.0 7 6.0 8 7.0 9 8.0 Name: A, Length: 10, dtype: float64
And multiple aggregations
In [8]: r.agg({'A': ['mean', 'std'], ...: 'B': ['mean', 'std']}) ...: Out[8]: A B mean std mean std 0 NaN NaN NaN NaN 1 NaN NaN NaN NaN 2 1.0 1.0 0.237722 1.327364 3 2.0 1.0 -0.023640 1.335505 4 3.0 1.0 0.133155 1.143778 5 4.0 1.0 -0.048693 0.835747 6 5.0 1.0 0.342054 0.920379 7 6.0 1.0 0.370076 0.871850 8 7.0 1.0 0.079587 0.750099 9 8.0 1.0 -0.954504 1.162285 [10 rows x 4 columns]
Series.rename and NDFrame.rename_axis can now take a scalar or list-like argument for altering the Series or axis name, in addition to their old behaviors of altering labels. (GH9494, GH11965)
Series.rename
NDFrame.rename_axis
In [9]: s = pd.Series(np.random.randn(5)) In [10]: s.rename('newname') Out[10]: 0 1.150036 1 0.991946 2 0.953324 3 -2.021255 4 -0.334077 Name: newname, Length: 5, dtype: float64
In [11]: df = pd.DataFrame(np.random.randn(5, 2)) In [12]: (df.rename_axis("indexname") ....: .rename_axis("columns_name", axis="columns")) ....: Out[12]: columns_name 0 1 indexname 0 0.002118 0.405453 1 0.289092 1.321158 2 -1.546906 -0.202646 3 -0.655969 0.193421 4 0.553439 1.318152 [5 rows x 2 columns]
The new functionality works well in method chains. Previously these methods only accepted functions or dicts mapping a label to a new label. This continues to work as before for function or dict-like values.
A RangeIndex has been added to the Int64Index sub-classes to support a memory saving alternative for common use cases. This has a similar implementation to the python range object (xrange in python 2), in that it only stores the start, stop, and step values for the index. It will transparently interact with the user API, converting to Int64Index if needed.
range
xrange
This will now be the default constructed index for NDFrame objects, rather than previous an Int64Index. (GH939, GH12070, GH12071, GH12109, GH12888)
NDFrame
In [3]: s = pd.Series(range(1000)) In [4]: s.index Out[4]: Int64Index([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... 990, 991, 992, 993, 994, 995, 996, 997, 998, 999], dtype='int64', length=1000) In [6]: s.index.nbytes Out[6]: 8000
In [13]: s = pd.Series(range(1000)) In [14]: s.index Out[14]: RangeIndex(start=0, stop=1000, step=1) In [15]: s.index.nbytes Out[15]: 128
The .str.extract method takes a regular expression with capture groups, finds the first match in each subject string, and returns the contents of the capture groups (GH11386).
In v0.18.0, the expand argument was added to extract.
expand
extract
expand=False: it returns a Series, Index, or DataFrame, depending on the subject and regular expression pattern (same behavior as pre-0.18.0).
expand=False
Series
Index
DataFrame
expand=True: it always returns a DataFrame, which is more consistent and less confusing from the perspective of a user.
expand=True
Currently the default is expand=None which gives a FutureWarning and uses expand=False. To avoid this warning, please explicitly specify expand.
expand=None
FutureWarning
In [1]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=None) FutureWarning: currently extract(expand=None) means expand=False (return Index/Series/DataFrame) but in a future version of pandas this will be changed to expand=True (return DataFrame) Out[1]: 0 1 1 2 2 NaN dtype: object
Extracting a regular expression with one group returns a Series if expand=False.
In [16]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=False) Out[16]: 0 1 1 2 2 NaN Length: 3, dtype: object
It returns a DataFrame with one column if expand=True.
In [17]: pd.Series(['a1', 'b2', 'c3']).str.extract(r'[ab](\d)', expand=True) Out[17]: 0 0 1 1 2 2 NaN [3 rows x 1 columns]
Calling on an Index with a regex with exactly one capture group returns an Index if expand=False.
In [18]: s = pd.Series(["a1", "b2", "c3"], ["A11", "B22", "C33"]) In [19]: s.index Out[19]: Index(['A11', 'B22', 'C33'], dtype='object') In [20]: s.index.str.extract("(?P<letter>[a-zA-Z])", expand=False) Out[20]: Index(['A', 'B', 'C'], dtype='object', name='letter')
In [21]: s.index.str.extract("(?P<letter>[a-zA-Z])", expand=True) Out[21]: letter 0 A 1 B 2 C [3 rows x 1 columns]
Calling on an Index with a regex with more than one capture group raises ValueError if expand=False.
ValueError
>>> s.index.str.extract("(?P<letter>[a-zA-Z])([0-9]+)", expand=False) ValueError: only one regex group is supported with Index
It returns a DataFrame if expand=True.
In [22]: s.index.str.extract("(?P<letter>[a-zA-Z])([0-9]+)", expand=True) Out[22]: letter 1 0 A 11 1 B 22 2 C 33 [3 rows x 2 columns]
In summary, extract(expand=True) always returns a DataFrame with a row for every subject string, and a column for every capture group.
extract(expand=True)
The .str.extractall method was added (GH11386). Unlike extract, which returns only the first match.
In [23]: s = pd.Series(["a1a2", "b1", "c1"], ["A", "B", "C"]) In [24]: s Out[24]: A a1a2 B b1 C c1 Length: 3, dtype: object In [25]: s.str.extract(r"(?P<letter>[ab])(?P<digit>\d)", expand=False) Out[25]: letter digit A a 1 B b 1 C NaN NaN [3 rows x 2 columns]
The extractall method returns all matches.
extractall
In [26]: s.str.extractall(r"(?P<letter>[ab])(?P<digit>\d)") Out[26]: letter digit match A 0 a 1 1 a 2 B 0 b 1 [3 rows x 2 columns]
The method .str.cat() concatenates the members of a Series. Before, if NaN values were present in the Series, calling .str.cat() on it would return NaN, unlike the rest of the Series.str.* API. This behavior has been amended to ignore NaN values by default. (GH11435).
.str.cat()
NaN
Series.str.*
A new, friendlier ValueError is added to protect against the mistake of supplying the sep as an arg, rather than as a kwarg. (GH11334).
sep
In [27]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(sep=' ') Out[27]: 'a b c' In [28]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(sep=' ', na_rep='?') Out[28]: 'a b ? c'
In [2]: pd.Series(['a', 'b', np.nan, 'c']).str.cat(' ') ValueError: Did you mean to supply a `sep` keyword?
DatetimeIndex, Timestamp, TimedeltaIndex, Timedelta have gained the .round(), .floor() and .ceil() method for datetimelike rounding, flooring and ceiling. (GH4314, GH11963)
DatetimeIndex
Timestamp
TimedeltaIndex
Timedelta
.round()
.floor()
.ceil()
Naive datetimes
In [29]: dr = pd.date_range('20130101 09:12:56.1234', periods=3) In [30]: dr Out[30]: DatetimeIndex(['2013-01-01 09:12:56.123400', '2013-01-02 09:12:56.123400', '2013-01-03 09:12:56.123400'], dtype='datetime64[ns]', freq='D') In [31]: dr.round('s') Out[31]: DatetimeIndex(['2013-01-01 09:12:56', '2013-01-02 09:12:56', '2013-01-03 09:12:56'], dtype='datetime64[ns]', freq=None) # Timestamp scalar In [32]: dr[0] Out[32]: Timestamp('2013-01-01 09:12:56.123400', freq='D') In [33]: dr[0].round('10s') Out[33]: Timestamp('2013-01-01 09:13:00')
Tz-aware are rounded, floored and ceiled in local times
In [34]: dr = dr.tz_localize('US/Eastern') In [35]: dr Out[35]: DatetimeIndex(['2013-01-01 09:12:56.123400-05:00', '2013-01-02 09:12:56.123400-05:00', '2013-01-03 09:12:56.123400-05:00'], dtype='datetime64[ns, US/Eastern]', freq='D') In [36]: dr.round('s') Out[36]: DatetimeIndex(['2013-01-01 09:12:56-05:00', '2013-01-02 09:12:56-05:00', '2013-01-03 09:12:56-05:00'], dtype='datetime64[ns, US/Eastern]', freq=None)
Timedeltas
In [37]: t = pd.timedelta_range('1 days 2 hr 13 min 45 us', periods=3, freq='d') In [38]: t Out[38]: TimedeltaIndex(['1 days 02:13:00.000045', '2 days 02:13:00.000045', '3 days 02:13:00.000045'], dtype='timedelta64[ns]', freq='D') In [39]: t.round('10min') Out[39]: TimedeltaIndex(['1 days 02:10:00', '2 days 02:10:00', '3 days 02:10:00'], dtype='timedelta64[ns]', freq=None) # Timedelta scalar In [40]: t[0] Out[40]: Timedelta('1 days 02:13:00.000045') In [41]: t[0].round('2h') Out[41]: Timedelta('1 days 02:00:00')
In addition, .round(), .floor() and .ceil() will be available through the .dt accessor of Series.
.dt
In [42]: s = pd.Series(dr) In [43]: s Out[43]: 0 2013-01-01 09:12:56.123400-05:00 1 2013-01-02 09:12:56.123400-05:00 2 2013-01-03 09:12:56.123400-05:00 Length: 3, dtype: datetime64[ns, US/Eastern] In [44]: s.dt.round('D') Out[44]: 0 2013-01-01 00:00:00-05:00 1 2013-01-02 00:00:00-05:00 2 2013-01-03 00:00:00-05:00 Length: 3, dtype: datetime64[ns, US/Eastern]
Integers in FloatIndex, e.g. 1., are now formatted with a decimal point and a 0 digit, e.g. 1.0 (GH11713) This change not only affects the display to the console, but also the output of IO methods like .to_csv or .to_html.
FloatIndex
0
1.0
.to_csv
.to_html
In [2]: s = pd.Series([1, 2, 3], index=np.arange(3.)) In [3]: s Out[3]: 0 1 1 2 2 3 dtype: int64 In [4]: s.index Out[4]: Float64Index([0.0, 1.0, 2.0], dtype='float64') In [5]: print(s.to_csv(path=None)) 0,1 1,2 2,3
In [45]: s = pd.Series([1, 2, 3], index=np.arange(3.)) In [46]: s Out[46]: 0.0 1 1.0 2 2.0 3 Length: 3, dtype: int64 In [47]: s.index Out[47]: Float64Index([0.0, 1.0, 2.0], dtype='float64') In [48]: print(s.to_csv(path_or_buf=None, header=False)) 0.0,1 1.0,2 2.0,3
When a DataFrame’s slice is updated with a new slice of the same dtype, the dtype of the DataFrame will now remain the same. (GH10503)
In [5]: df = pd.DataFrame({'a': [0, 1, 1], 'b': pd.Series([100, 200, 300], dtype='uint32')}) In [7]: df.dtypes Out[7]: a int64 b uint32 dtype: object In [8]: ix = df['a'] == 1 In [9]: df.loc[ix, 'b'] = df.loc[ix, 'b'] In [11]: df.dtypes Out[11]: a int64 b int64 dtype: object
In [49]: df = pd.DataFrame({'a': [0, 1, 1], ....: 'b': pd.Series([100, 200, 300], dtype='uint32')}) ....: In [50]: df.dtypes Out[50]: a int64 b uint32 Length: 2, dtype: object In [51]: ix = df['a'] == 1 In [52]: df.loc[ix, 'b'] = df.loc[ix, 'b'] In [53]: df.dtypes Out[53]: a int64 b uint32 Length: 2, dtype: object
When a DataFrame’s integer slice is partially updated with a new slice of floats that could potentially be down-casted to integer without losing precision, the dtype of the slice will be set to float instead of integer.
In [4]: df = pd.DataFrame(np.array(range(1,10)).reshape(3,3), columns=list('abc'), index=[[4,4,8], [8,10,12]]) In [5]: df Out[5]: a b c 4 8 1 2 3 10 4 5 6 8 12 7 8 9 In [7]: df.ix[4, 'c'] = np.array([0., 1.]) In [8]: df Out[8]: a b c 4 8 1 2 0 10 4 5 1 8 12 7 8 9
In [54]: df = pd.DataFrame(np.array(range(1,10)).reshape(3,3), ....: columns=list('abc'), ....: index=[[4,4,8], [8,10,12]]) ....: In [55]: df Out[55]: a b c 4 8 1 2 3 10 4 5 6 8 12 7 8 9 [3 rows x 3 columns] In [56]: df.loc[4, 'c'] = np.array([0., 1.]) In [57]: df Out[57]: a b c 4 8 1 2 0.0 10 4 5 1.0 8 12 7 8 9.0 [3 rows x 3 columns]
In a future version of pandas, we will be deprecating Panel and other > 2 ndim objects. In order to provide for continuity, all NDFrame objects have gained the .to_xarray() method in order to convert to xarray objects, which has a pandas-like interface for > 2 ndim. (GH11972)
Panel
xarray
See the xarray full-documentation here.
In [1]: p = Panel(np.arange(2*3*4).reshape(2,3,4)) In [2]: p.to_xarray() Out[2]: <xarray.DataArray (items: 2, major_axis: 3, minor_axis: 4)> array([[[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11]], [[12, 13, 14, 15], [16, 17, 18, 19], [20, 21, 22, 23]]]) Coordinates: * items (items) int64 0 1 * major_axis (major_axis) int64 0 1 2 * minor_axis (minor_axis) int64 0 1 2 3
DataFrame has gained a ._repr_latex_() method in order to allow for conversion to latex in a ipython/jupyter notebook using nbconvert. (GH11778)
._repr_latex_()
Note that this must be activated by setting the option pd.display.latex.repr=True (GH12182)
pd.display.latex.repr=True
For example, if you have a jupyter notebook you plan to convert to latex using nbconvert, place the statement pd.display.latex.repr=True in the first cell to have the contained DataFrame output also stored as latex.
The options display.latex.escape and display.latex.longtable have also been added to the configuration and are used automatically by the to_latex method. See the available options docs for more info.
display.latex.escape
display.latex.longtable
to_latex
read_sas has gained the ability to read SAS7BDAT files, including compressed files. The files can be read in entirety, or incrementally. For full details see here. (GH4052)
Handle truncated floats in SAS xport files (GH11713)
Added option to hide index in Series.to_string (GH11729)
Series.to_string
read_excel now supports s3 urls of the format s3://bucketname/filename (GH11447)
read_excel
s3://bucketname/filename
add support for AWS_S3_HOST env variable when reading from s3 (GH12198)
AWS_S3_HOST
A simple version of Panel.round() is now implemented (GH11763)
Panel.round()
For Python 3.x, round(DataFrame), round(Series), round(Panel) will work (GH11763)
round(DataFrame)
round(Series)
round(Panel)
sys.getsizeof(obj) returns the memory usage of a pandas object, including the values it contains (GH11597)
sys.getsizeof(obj)
Series gained an is_unique attribute (GH11946)
is_unique
DataFrame.quantile and Series.quantile now accept interpolation keyword (GH10174).
DataFrame.quantile
Series.quantile
interpolation
Added DataFrame.style.format for more flexible formatting of cell values (GH11692)
DataFrame.style.format
DataFrame.select_dtypes now allows the np.float16 type code (GH11990)
DataFrame.select_dtypes
np.float16
pivot_table() now accepts most iterables for the values parameter (GH12017)
pivot_table()
values
Added Google BigQuery service account authentication support, which enables authentication on remote servers. (GH11881, GH12572). For further details see here
BigQuery
HDFStore is now iterable: for k in store is equivalent to for k in store.keys() (GH12221).
HDFStore
for k in store
for k in store.keys()
Add missing methods/fields to .dt for Period (GH8848)
Period
The entire code base has been PEP-ified (GH12096)
PEP
the leading white spaces have been removed from the output of .to_string(index=False) method (GH11833)
.to_string(index=False)
the out parameter has been removed from the Series.round() method. (GH11763)
out
Series.round()
DataFrame.round() leaves non-numeric columns unchanged in its return, rather than raises. (GH11885)
DataFrame.round()
DataFrame.head(0) and DataFrame.tail(0) return empty frames, rather than self. (GH11937)
DataFrame.head(0)
DataFrame.tail(0)
self
Series.head(0) and Series.tail(0) return empty series, rather than self. (GH11937)
Series.head(0)
Series.tail(0)
to_msgpack and read_msgpack encoding now defaults to 'utf-8'. (GH12170)
to_msgpack
read_msgpack
'utf-8'
the order of keyword arguments to text file parsing functions (.read_csv(), .read_table(), .read_fwf()) changed to group related arguments. (GH11555)
.read_csv()
.read_table()
.read_fwf()
NaTType.isoformat now returns the string 'NaT to allow the result to be passed to the constructor of Timestamp. (GH12300)
NaTType.isoformat
'NaT
NaT and Timedelta have expanded arithmetic operations, which are extended to Series arithmetic where applicable. Operations defined for datetime64[ns] or timedelta64[ns] are now also defined for NaT (GH11564).
NaT
datetime64[ns]
timedelta64[ns]
NaT now supports arithmetic operations with integers and floats.
In [58]: pd.NaT * 1 Out[58]: NaT In [59]: pd.NaT * 1.5 Out[59]: NaT In [60]: pd.NaT / 2 Out[60]: NaT In [61]: pd.NaT * np.nan Out[61]: NaT
NaT defines more arithmetic operations with datetime64[ns] and timedelta64[ns].
In [62]: pd.NaT / pd.NaT Out[62]: nan In [63]: pd.Timedelta('1s') / pd.NaT Out[63]: nan
NaT may represent either a datetime64[ns] null or a timedelta64[ns] null. Given the ambiguity, it is treated as a timedelta64[ns], which allows more operations to succeed.
In [64]: pd.NaT + pd.NaT Out[64]: NaT # same as In [65]: pd.Timedelta('1s') + pd.Timedelta('1s') Out[65]: Timedelta('0 days 00:00:02')
as opposed to
In [3]: pd.Timestamp('19900315') + pd.Timestamp('19900315') TypeError: unsupported operand type(s) for +: 'Timestamp' and 'Timestamp'
However, when wrapped in a Series whose dtype is datetime64[ns] or timedelta64[ns], the dtype information is respected.
dtype
In [1]: pd.Series([pd.NaT], dtype='<M8[ns]') + pd.Series([pd.NaT], dtype='<M8[ns]') TypeError: can only operate on a datetimes for subtraction, but the operator [__add__] was passed
In [66]: pd.Series([pd.NaT], dtype='<m8[ns]') + pd.Series([pd.NaT], dtype='<m8[ns]') Out[66]: 0 NaT Length: 1, dtype: timedelta64[ns]
Timedelta division by floats now works.
floats
In [67]: pd.Timedelta('1s') / 2.0 Out[67]: Timedelta('0 days 00:00:00.500000')
Subtraction by Timedelta in a Series by a Timestamp works (GH11925)
In [68]: ser = pd.Series(pd.timedelta_range('1 day', periods=3)) In [69]: ser Out[69]: 0 1 days 1 2 days 2 3 days Length: 3, dtype: timedelta64[ns] In [70]: pd.Timestamp('2012-01-01') - ser Out[70]: 0 2011-12-31 1 2011-12-30 2 2011-12-29 Length: 3, dtype: datetime64[ns]
NaT.isoformat() now returns 'NaT'. This change allows allows pd.Timestamp to rehydrate any timestamp like object from its isoformat (GH12300).
NaT.isoformat()
'NaT'
pd.Timestamp
Forward incompatible changes in msgpack writing format were made over 0.17.0 and 0.18.0; older versions of pandas cannot read files packed by newer versions (GH12129, GH10527)
msgpack
Bugs in to_msgpack and read_msgpack introduced in 0.17.0 and fixed in 0.18.0, caused files packed in Python 2 unreadable by Python 3 (GH12142). The following table describes the backward and forward compat of msgpacks.
Packed with
Can be unpacked with
pre-0.17 / Python 2
any
pre-0.17 / Python 3
0.17 / Python 2
==0.17 / Python 2
>=0.18 / any Python
0.17 / Python 3
0.18
>= 0.18
0.18.0 is backward-compatible for reading files packed by older versions, except for files packed with 0.17 in Python 2, in which case only they can only be unpacked in Python 2.
Series.rank and DataFrame.rank now have the same signature (GH11759)
Series.rank
DataFrame.rank
Previous signature
In [3]: pd.Series([0,1]).rank(method='average', na_option='keep', ascending=True, pct=False) Out[3]: 0 1 1 2 dtype: float64 In [4]: pd.DataFrame([0,1]).rank(axis=0, numeric_only=None, method='average', na_option='keep', ascending=True, pct=False) Out[4]: 0 0 1 1 2
New signature
In [71]: pd.Series([0,1]).rank(axis=0, method='average', numeric_only=None, ....: na_option='keep', ascending=True, pct=False) ....: Out[71]: 0 1.0 1 2.0 Length: 2, dtype: float64 In [72]: pd.DataFrame([0,1]).rank(axis=0, method='average', numeric_only=None, ....: na_option='keep', ascending=True, pct=False) ....: Out[72]: 0 0 1.0 1 2.0 [2 rows x 1 columns]
In previous versions, the behavior of the QuarterBegin offset was inconsistent depending on the date when the n parameter was 0. (GH11406)
n
The general semantics of anchored offsets for n=0 is to not move the date when it is an anchor point (e.g., a quarter start date), and otherwise roll forward to the next anchor point.
n=0
In [73]: d = pd.Timestamp('2014-02-01') In [74]: d Out[74]: Timestamp('2014-02-01 00:00:00') In [75]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2) Out[75]: Timestamp('2014-02-01 00:00:00') In [76]: d + pd.offsets.QuarterBegin(n=0, startingMonth=1) Out[76]: Timestamp('2014-04-01 00:00:00')
For the QuarterBegin offset in previous versions, the date would be rolled backwards if date was in the same month as the quarter start date.
QuarterBegin
In [3]: d = pd.Timestamp('2014-02-15') In [4]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2) Out[4]: Timestamp('2014-02-01 00:00:00')
This behavior has been corrected in version 0.18.0, which is consistent with other anchored offsets like MonthBegin and YearBegin.
MonthBegin
YearBegin
In [77]: d = pd.Timestamp('2014-02-15') In [78]: d + pd.offsets.QuarterBegin(n=0, startingMonth=2) Out[78]: Timestamp('2014-05-01 00:00:00')
Like the change in the window functions API above, .resample(...) is changing to have a more groupby-like API. (GH11732, GH12702, GH12202, GH12332, GH12334, GH12348, GH12448).
.resample(...)
In [79]: np.random.seed(1234) In [80]: df = pd.DataFrame(np.random.rand(10,4), ....: columns=list('ABCD'), ....: index=pd.date_range('2010-01-01 09:00:00', ....: periods=10, freq='s')) ....: In [81]: df Out[81]: A B C D 2010-01-01 09:00:00 0.191519 0.622109 0.437728 0.785359 2010-01-01 09:00:01 0.779976 0.272593 0.276464 0.801872 2010-01-01 09:00:02 0.958139 0.875933 0.357817 0.500995 2010-01-01 09:00:03 0.683463 0.712702 0.370251 0.561196 2010-01-01 09:00:04 0.503083 0.013768 0.772827 0.882641 2010-01-01 09:00:05 0.364886 0.615396 0.075381 0.368824 2010-01-01 09:00:06 0.933140 0.651378 0.397203 0.788730 2010-01-01 09:00:07 0.316836 0.568099 0.869127 0.436173 2010-01-01 09:00:08 0.802148 0.143767 0.704261 0.704581 2010-01-01 09:00:09 0.218792 0.924868 0.442141 0.909316 [10 rows x 4 columns]
Previous API:
You would write a resampling operation that immediately evaluates. If a how parameter was not provided, it would default to how='mean'.
how
how='mean'
In [6]: df.resample('2s') Out[6]: A B C D 2010-01-01 09:00:00 0.485748 0.447351 0.357096 0.793615 2010-01-01 09:00:02 0.820801 0.794317 0.364034 0.531096 2010-01-01 09:00:04 0.433985 0.314582 0.424104 0.625733 2010-01-01 09:00:06 0.624988 0.609738 0.633165 0.612452 2010-01-01 09:00:08 0.510470 0.534317 0.573201 0.806949
You could also specify a how directly
In [7]: df.resample('2s', how='sum') Out[7]: A B C D 2010-01-01 09:00:00 0.971495 0.894701 0.714192 1.587231 2010-01-01 09:00:02 1.641602 1.588635 0.728068 1.062191 2010-01-01 09:00:04 0.867969 0.629165 0.848208 1.251465 2010-01-01 09:00:06 1.249976 1.219477 1.266330 1.224904 2010-01-01 09:00:08 1.020940 1.068634 1.146402 1.613897
New API:
Now, you can write .resample(..) as a 2-stage operation like .groupby(...), which yields a Resampler.
.resample(..)
.groupby(...)
Resampler
In [82]: r = df.resample('2s') In [83]: r Out[83]: <pandas.core.resample.DatetimeIndexResampler object at 0x7f657df115c0>
You can then use this object to perform operations. These are downsampling operations (going from a higher frequency to a lower one).
In [84]: r.mean() Out[84]: A B C D 2010-01-01 09:00:00 0.485748 0.447351 0.357096 0.793615 2010-01-01 09:00:02 0.820801 0.794317 0.364034 0.531096 2010-01-01 09:00:04 0.433985 0.314582 0.424104 0.625733 2010-01-01 09:00:06 0.624988 0.609738 0.633165 0.612452 2010-01-01 09:00:08 0.510470 0.534317 0.573201 0.806949 [5 rows x 4 columns]
In [85]: r.sum() Out[85]: A B C D 2010-01-01 09:00:00 0.971495 0.894701 0.714192 1.587231 2010-01-01 09:00:02 1.641602 1.588635 0.728068 1.062191 2010-01-01 09:00:04 0.867969 0.629165 0.848208 1.251465 2010-01-01 09:00:06 1.249976 1.219477 1.266330 1.224904 2010-01-01 09:00:08 1.020940 1.068634 1.146402 1.613897 [5 rows x 4 columns]
Furthermore, resample now supports getitem operations to perform the resample on specific columns.
getitem
In [86]: r[['A','C']].mean() Out[86]: A C 2010-01-01 09:00:00 0.485748 0.357096 2010-01-01 09:00:02 0.820801 0.364034 2010-01-01 09:00:04 0.433985 0.424104 2010-01-01 09:00:06 0.624988 0.633165 2010-01-01 09:00:08 0.510470 0.573201 [5 rows x 2 columns]
and .aggregate type operations.
.aggregate
In [87]: r.agg({'A' : 'mean', 'B' : 'sum'}) Out[87]: A B 2010-01-01 09:00:00 0.485748 0.894701 2010-01-01 09:00:02 0.820801 1.588635 2010-01-01 09:00:04 0.433985 0.629165 2010-01-01 09:00:06 0.624988 1.219477 2010-01-01 09:00:08 0.510470 1.068634 [5 rows x 2 columns]
These accessors can of course, be combined
In [88]: r[['A','B']].agg(['mean','sum']) Out[88]: A B mean sum mean sum 2010-01-01 09:00:00 0.485748 0.971495 0.447351 0.894701 2010-01-01 09:00:02 0.820801 1.641602 0.794317 1.588635 2010-01-01 09:00:04 0.433985 0.867969 0.314582 0.629165 2010-01-01 09:00:06 0.624988 1.249976 0.609738 1.219477 2010-01-01 09:00:08 0.510470 1.020940 0.534317 1.068634 [5 rows x 4 columns]
Upsampling operations take you from a lower frequency to a higher frequency. These are now performed with the Resampler objects with backfill(), ffill(), fillna() and asfreq() methods.
backfill()
ffill()
fillna()
asfreq()
In [89]: s = pd.Series(np.arange(5, dtype='int64'), ....: index=pd.date_range('2010-01-01', periods=5, freq='Q')) ....: In [90]: s Out[90]: 2010-03-31 0 2010-06-30 1 2010-09-30 2 2010-12-31 3 2011-03-31 4 Freq: Q-DEC, Length: 5, dtype: int64
Previously
In [6]: s.resample('M', fill_method='ffill') Out[6]: 2010-03-31 0 2010-04-30 0 2010-05-31 0 2010-06-30 1 2010-07-31 1 2010-08-31 1 2010-09-30 2 2010-10-31 2 2010-11-30 2 2010-12-31 3 2011-01-31 3 2011-02-28 3 2011-03-31 4 Freq: M, dtype: int64
New API
In [91]: s.resample('M').ffill() Out[91]: 2010-03-31 0 2010-04-30 0 2010-05-31 0 2010-06-30 1 2010-07-31 1 2010-08-31 1 2010-09-30 2 2010-10-31 2 2010-11-30 2 2010-12-31 3 2011-01-31 3 2011-02-28 3 2011-03-31 4 Freq: M, Length: 13, dtype: int64
Note
In the new API, you can either downsample OR upsample. The prior implementation would allow you to pass an aggregator function (like mean) even though you were upsampling, providing a bit of confusion.
mean
This new API for resample includes some internal changes for the prior-to-0.18.0 API, to work with a deprecation warning in most cases, as the resample operation returns a deferred object. We can intercept operations and just do what the (pre 0.18.0) API did (with a warning). Here is a typical use case:
In [4]: r = df.resample('2s') In [6]: r*10 pandas/tseries/resample.py:80: FutureWarning: .resample() is now a deferred operation use .resample(...).mean() instead of .resample(...) Out[6]: A B C D 2010-01-01 09:00:00 4.857476 4.473507 3.570960 7.936154 2010-01-01 09:00:02 8.208011 7.943173 3.640340 5.310957 2010-01-01 09:00:04 4.339846 3.145823 4.241039 6.257326 2010-01-01 09:00:06 6.249881 6.097384 6.331650 6.124518 2010-01-01 09:00:08 5.104699 5.343172 5.732009 8.069486
However, getting and assignment operations directly on a Resampler will raise a ValueError:
In [7]: r.iloc[0] = 5 ValueError: .resample() is now a deferred operation use .resample(...).mean() instead of .resample(...)
There is a situation where the new API can not perform all the operations when using original code. This code is intending to resample every 2s, take the mean AND then take the min of those results.
min
In [4]: df.resample('2s').min() Out[4]: A 0.433985 B 0.314582 C 0.357096 D 0.531096 dtype: float64
The new API will:
In [92]: df.resample('2s').min() Out[92]: A B C D 2010-01-01 09:00:00 0.191519 0.272593 0.276464 0.785359 2010-01-01 09:00:02 0.683463 0.712702 0.357817 0.500995 2010-01-01 09:00:04 0.364886 0.013768 0.075381 0.368824 2010-01-01 09:00:06 0.316836 0.568099 0.397203 0.436173 2010-01-01 09:00:08 0.218792 0.143767 0.442141 0.704581 [5 rows x 4 columns]
The good news is the return dimensions will differ between the new API and the old API, so this should loudly raise an exception.
To replicate the original operation
In [93]: df.resample('2s').mean().min() Out[93]: A 0.433985 B 0.314582 C 0.357096 D 0.531096 Length: 4, dtype: float64
In prior versions, new columns assignments in an eval expression resulted in an inplace change to the DataFrame. (GH9297, GH8664, GH10486)
eval
In [94]: df = pd.DataFrame({'a': np.linspace(0, 10, 5), 'b': range(5)}) In [95]: df Out[95]: a b 0 0.0 0 1 2.5 1 2 5.0 2 3 7.5 3 4 10.0 4 [5 rows x 2 columns]
In [12]: df.eval('c = a + b') FutureWarning: eval expressions containing an assignment currentlydefault to operating inplace. This will change in a future version of pandas, use inplace=True to avoid this warning. In [13]: df Out[13]: a b c 0 0.0 0 0.0 1 2.5 1 3.5 2 5.0 2 7.0 3 7.5 3 10.5 4 10.0 4 14.0
In version 0.18.0, a new inplace keyword was added to choose whether the assignment should be done inplace or return a copy.
inplace
In [96]: df Out[96]: a b c 0 0.0 0 0.0 1 2.5 1 3.5 2 5.0 2 7.0 3 7.5 3 10.5 4 10.0 4 14.0 [5 rows x 3 columns] In [97]: df.eval('d = c - b', inplace=False) Out[97]: a b c d 0 0.0 0 0.0 0.0 1 2.5 1 3.5 2.5 2 5.0 2 7.0 5.0 3 7.5 3 10.5 7.5 4 10.0 4 14.0 10.0 [5 rows x 4 columns] In [98]: df Out[98]: a b c 0 0.0 0 0.0 1 2.5 1 3.5 2 5.0 2 7.0 3 7.5 3 10.5 4 10.0 4 14.0 [5 rows x 3 columns] In [99]: df.eval('d = c - b', inplace=True) In [100]: df Out[100]: a b c d 0 0.0 0 0.0 0.0 1 2.5 1 3.5 2.5 2 5.0 2 7.0 5.0 3 7.5 3 10.5 7.5 4 10.0 4 14.0 10.0 [5 rows x 4 columns]
For backwards compatibility, inplace defaults to True if not specified. This will change in a future version of pandas. If your code depends on an inplace assignment you should update to explicitly set inplace=True
True
inplace=True
The inplace keyword parameter was also added the query method.
query
In [101]: df.query('a > 5') Out[101]: a b c d 3 7.5 3 10.5 7.5 4 10.0 4 14.0 10.0 [2 rows x 4 columns] In [102]: df.query('a > 5', inplace=True) In [103]: df Out[103]: a b c d 3 7.5 3 10.5 7.5 4 10.0 4 14.0 10.0 [2 rows x 4 columns]
Note that the default value for inplace in a query is False, which is consistent with prior versions.
False
eval has also been updated to allow multi-line expressions for multiple assignments. These expressions will be evaluated one at a time in order. Only assignments are valid for multi-line expressions.
In [104]: df Out[104]: a b c d 3 7.5 3 10.5 7.5 4 10.0 4 14.0 10.0 [2 rows x 4 columns] In [105]: df.eval(""" .....: e = d + a .....: f = e - 22 .....: g = f / 2.0""", inplace=True) .....: In [106]: df Out[106]: a b c d e f g 3 7.5 3 10.5 7.5 15.0 -7.0 -3.5 4 10.0 4 14.0 10.0 20.0 -2.0 -1.0 [2 rows x 7 columns]
DataFrame.between_time and Series.between_time now only parse a fixed set of time strings. Parsing of date strings is no longer supported and raises a ValueError. (GH11818)
DataFrame.between_time
Series.between_time
In [107]: s = pd.Series(range(10), pd.date_range('2015-01-01', freq='H', periods=10)) In [108]: s.between_time("7:00am", "9:00am") --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-108-c90c8ac4f6a1> in <module> ----> 1 s.between_time("7:00am", "9:00am") ~/scipy/pandas/pandas/core/generic.py in between_time(self, start_time, end_time, include_start, include_end, axis) 7545 end_time, 7546 include_start=include_start, -> 7547 include_end=include_end, 7548 ) 7549 except AttributeError: ~/scipy/pandas/pandas/core/indexes/datetimes.py in indexer_between_time(self, start_time, end_time, include_start, include_end) 964 indexer_at_time, DataFrame.between_time 965 """ --> 966 start_time = tools.to_time(start_time) 967 end_time = tools.to_time(end_time) 968 time_micros = self._get_time_micros() ~/scipy/pandas/pandas/core/tools/datetimes.py in to_time(arg, format, infer_time_format, errors) 1036 return _convert_listlike(arg, format) 1037 -> 1038 return _convert_listlike(np.array([arg]), format)[0] ~/scipy/pandas/pandas/core/tools/datetimes.py in _convert_listlike(arg, format) 1016 times.append(time_object) 1017 elif errors == "raise": -> 1018 raise ValueError(f"Cannot convert arg {arg} to a time") 1019 elif errors == "ignore": 1020 return arg ValueError: Cannot convert arg ['7:00am'] to a time
This will now raise.
In [2]: s.between_time('20150101 07:00:00','20150101 09:00:00') ValueError: Cannot convert arg ['20150101 07:00:00'] to a time.
.memory_usage() now includes values in the index, as does memory_usage in .info() (GH11597)
.memory_usage()
.info()
DataFrame.to_latex() now supports non-ascii encodings (eg utf-8) in Python 2 with the parameter encoding (GH7061)
DataFrame.to_latex()
utf-8
encoding
pandas.merge() and DataFrame.merge() will show a specific error message when trying to merge with an object that is not of type DataFrame or a subclass (GH12081)
pandas.merge()
DataFrame.merge()
DataFrame.unstack and Series.unstack now take fill_value keyword to allow direct replacement of missing values when an unstack results in missing values in the resulting DataFrame. As an added benefit, specifying fill_value will preserve the data type of the original stacked data. (GH9746)
DataFrame.unstack
Series.unstack
fill_value
As part of the new API for window functions and resampling, aggregation functions have been clarified, raising more informative error messages on invalid aggregations. (GH9052). A full set of examples are presented in groupby.
Statistical functions for NDFrame objects (like sum(), mean(), min()) will now raise if non-numpy-compatible arguments are passed in for **kwargs (GH12301)
sum(), mean(), min()
**kwargs
.to_latex and .to_html gain a decimal parameter like .to_csv; the default is '.' (GH12031)
.to_latex
decimal
'.'
More helpful error message when constructing a DataFrame with empty data but with indices (GH8020)
.describe() will now properly handle bool dtype as a categorical (GH6625)
.describe()
More helpful error message with an invalid .transform with user defined input (GH10165)
.transform
Exponentially weighted functions now allow specifying alpha directly (GH10789) and raise ValueError if parameters violate 0 < alpha <= 1 (GH12492)
0 < alpha <= 1
The functions pd.rolling_*, pd.expanding_*, and pd.ewm* are deprecated and replaced by the corresponding method call. Note that the new suggested syntax includes all of the arguments (even if default) (GH11603)
pd.rolling_*
pd.expanding_*
pd.ewm*
In [1]: s = pd.Series(range(3)) In [2]: pd.rolling_mean(s,window=2,min_periods=1) FutureWarning: pd.rolling_mean is deprecated for Series and will be removed in a future version, replace with Series.rolling(min_periods=1,window=2,center=False).mean() Out[2]: 0 0.0 1 0.5 2 1.5 dtype: float64 In [3]: pd.rolling_cov(s, s, window=2) FutureWarning: pd.rolling_cov is deprecated for Series and will be removed in a future version, replace with Series.rolling(window=2).cov(other=<Series>) Out[3]: 0 NaN 1 0.5 2 0.5 dtype: float64
The freq and how arguments to the .rolling, .expanding, and .ewm (new) functions are deprecated, and will be removed in a future version. You can simply resample the input prior to creating a window function. (GH11603).
freq
.rolling
.expanding
.ewm
For example, instead of s.rolling(window=5,freq='D').max() to get the max value on a rolling 5 Day window, one could use s.resample('D').mean().rolling(window=5).max(), which first resamples the data to daily data, then provides a rolling 5 day window.
s.rolling(window=5,freq='D').max()
s.resample('D').mean().rolling(window=5).max()
pd.tseries.frequencies.get_offset_name function is deprecated. Use offset’s .freqstr property as alternative (GH11192)
pd.tseries.frequencies.get_offset_name
.freqstr
pandas.stats.fama_macbeth routines are deprecated and will be removed in a future version (GH6077)
pandas.stats.fama_macbeth
pandas.stats.ols, pandas.stats.plm and pandas.stats.var routines are deprecated and will be removed in a future version (GH6077)
pandas.stats.ols
pandas.stats.plm
pandas.stats.var
show a FutureWarning rather than a DeprecationWarning on using long-time deprecated syntax in HDFStore.select, where the where clause is not a string-like (GH12027)
DeprecationWarning
HDFStore.select
where
The pandas.options.display.mpl_style configuration has been deprecated and will be removed in a future version of pandas. This functionality is better handled by matplotlib’s style sheets (GH11783).
pandas.options.display.mpl_style
In GH4892 indexing with floating point numbers on a non-Float64Index was deprecated (in version 0.14.0). In 0.18.0, this deprecation warning is removed and these will now raise a TypeError. (GH12165, GH12333)
Float64Index
In [109]: s = pd.Series([1, 2, 3], index=[4, 5, 6]) In [110]: s Out[110]: 4 1 5 2 6 3 Length: 3, dtype: int64 In [111]: s2 = pd.Series([1, 2, 3], index=list('abc')) In [112]: s2 Out[112]: a 1 b 2 c 3 Length: 3, dtype: int64
# this is label indexing In [2]: s[5.0] FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point Out[2]: 2 # this is positional indexing In [3]: s.iloc[1.0] FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point Out[3]: 2 # this is label indexing In [4]: s.loc[5.0] FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point Out[4]: 2 # .ix would coerce 1.0 to the positional 1, and index In [5]: s2.ix[1.0] = 10 FutureWarning: scalar indexers for index type Index should be integers and not floating point In [6]: s2 Out[6]: a 1 b 10 c 3 dtype: int64
For iloc, getting & setting via a float scalar will always raise.
In [3]: s.iloc[2.0] TypeError: cannot do label indexing on <class 'pandas.indexes.numeric.Int64Index'> with these indexers [2.0] of <type 'float'>
Other indexers will coerce to a like integer for both getting and setting. The FutureWarning has been dropped for .loc, .ix and [].
.loc
.ix
[]
In [113]: s[5.0] Out[113]: 2 In [114]: s.loc[5.0] Out[114]: 2
and setting
In [115]: s_copy = s.copy() In [116]: s_copy[5.0] = 10 In [117]: s_copy Out[117]: 4 1 5 10 6 3 Length: 3, dtype: int64 In [118]: s_copy = s.copy() In [119]: s_copy.loc[5.0] = 10 In [120]: s_copy Out[120]: 4 1 5 10 6 3 Length: 3, dtype: int64
Positional setting with .ix and a float indexer will ADD this value to the index, rather than previously setting the value by position.
In [3]: s2.ix[1.0] = 10 In [4]: s2 Out[4]: a 1 b 2 c 3 1.0 10 dtype: int64
Slicing will also coerce integer-like floats to integers for a non-Float64Index.
In [121]: s.loc[5.0:6] Out[121]: 5 2 6 3 Length: 2, dtype: int64
Note that for floats that are NOT coercible to ints, the label based bounds will be excluded
In [122]: s.loc[5.1:6] Out[122]: 6 3 Length: 1, dtype: int64
Float indexing on a Float64Index is unchanged.
In [123]: s = pd.Series([1, 2, 3], index=np.arange(3.)) In [124]: s[1.0] Out[124]: 2 In [125]: s[1.0:2.5] Out[125]: 1.0 2 2.0 3 Length: 2, dtype: int64
Removal of rolling_corr_pairwise in favor of .rolling().corr(pairwise=True) (GH4950)
rolling_corr_pairwise
.rolling().corr(pairwise=True)
Removal of expanding_corr_pairwise in favor of .expanding().corr(pairwise=True) (GH4950)
expanding_corr_pairwise
.expanding().corr(pairwise=True)
Removal of DataMatrix module. This was not imported into the pandas namespace in any event (GH12111)
DataMatrix
Removal of cols keyword in favor of subset in DataFrame.duplicated() and DataFrame.drop_duplicates() (GH6680)
cols
subset
DataFrame.duplicated()
DataFrame.drop_duplicates()
Removal of the read_frame and frame_query (both aliases for pd.read_sql) and write_frame (alias of to_sql) functions in the pd.io.sql namespace, deprecated since 0.14.0 (GH6292).
read_frame
frame_query
pd.read_sql
write_frame
to_sql
pd.io.sql
Removal of the order keyword from .factorize() (GH6930)
order
.factorize()
Improved performance of andrews_curves (GH11534)
andrews_curves
Improved huge DatetimeIndex, PeriodIndex and TimedeltaIndex’s ops performance including NaT (GH10277)
PeriodIndex
Improved performance of pandas.concat (GH11958)
pandas.concat
Improved performance of StataReader (GH11591)
StataReader
Improved performance in construction of Categoricals with Series of datetimes containing NaT (GH12077)
Categoricals
Improved performance of ISO 8601 date parsing for dates without separators (GH11899), leading zeros (GH11871) and with white space preceding the time zone (GH9714)
Bug in GroupBy.size when data-frame is empty. (GH11699)
GroupBy.size
Bug in Period.end_time when a multiple of time period is requested (GH11738)
Period.end_time
Regression in .clip with tz-aware datetimes (GH11838)
.clip
Bug in date_range when the boundaries fell on the frequency (GH11804, GH12409)
date_range
Bug in consistency of passing nested dicts to .groupby(...).agg(...) (GH9052)
.groupby(...).agg(...)
Accept unicode in Timedelta constructor (GH11995)
Bug in value label reading for StataReader when reading incrementally (GH12014)
Bug in vectorized DateOffset when n parameter is 0 (GH11370)
DateOffset
Compat for numpy 1.11 w.r.t. NaT comparison changes (GH12049)
Bug in read_csv when reading from a StringIO in threads (GH11790)
read_csv
StringIO
Bug in not treating NaT as a missing value in datetimelikes when factorizing & with Categoricals (GH12077)
Bug in getitem when the values of a Series were tz-aware (GH12089)
Bug in Series.str.get_dummies when one of the variables was ‘name’ (GH12180)
Series.str.get_dummies
Bug in pd.concat while concatenating tz-aware NaT series. (GH11693, GH11755, GH12217)
pd.concat
Bug in pd.read_stata with version <= 108 files (GH12232)
pd.read_stata
Bug in Series.resample using a frequency of Nano when the index is a DatetimeIndex and contains non-zero nanosecond parts (GH12037)
Series.resample
Nano
Bug in resampling with .nunique and a sparse index (GH12352)
.nunique
Removed some compiler warnings (GH12471)
Work around compat issues with boto in python 3.5 (GH11915)
boto
Bug in NaT subtraction from Timestamp or DatetimeIndex with timezones (GH11718)
Bug in subtraction of Series of a single tz-aware Timestamp (GH12290)
Use compat iterators in PY2 to support .next() (GH12299)
.next()
Bug in Timedelta.round with negative values (GH11690)
Timedelta.round
Bug in .loc against CategoricalIndex may result in normal Index (GH11586)
CategoricalIndex
Bug in DataFrame.info when duplicated column names exist (GH11761)
DataFrame.info
Bug in .copy of datetime tz-aware objects (GH11794)
.copy
Bug in Series.apply and Series.map where timedelta64 was not boxed (GH11349)
Series.apply
Series.map
timedelta64
Bug in DataFrame.set_index() with tz-aware Series (GH12358)
DataFrame.set_index()
Bug in subclasses of DataFrame where AttributeError did not propagate (GH11808)
AttributeError
Bug groupby on tz-aware data where selection not returning Timestamp (GH11616)
Bug in pd.read_clipboard and pd.to_clipboard functions not supporting Unicode; upgrade included pyperclip to v1.5.15 (GH9263)
pd.read_clipboard
pd.to_clipboard
pyperclip
Bug in DataFrame.query containing an assignment (GH8664)
DataFrame.query
Bug in from_msgpack where __contains__() fails for columns of the unpacked DataFrame, if the DataFrame has object columns. (GH11880)
from_msgpack
__contains__()
Bug in .resample on categorical data with TimedeltaIndex (GH12169)
Bug in timezone info lost when broadcasting scalar datetime to DataFrame (GH11682)
Bug in Index creation from Timestamp with mixed tz coerces to UTC (GH11488)
Bug in to_numeric where it does not raise if input is more than one dimension (GH11776)
to_numeric
Bug in parsing timezone offset strings with non-zero minutes (GH11708)
Bug in df.plot using incorrect colors for bar plots under matplotlib 1.5+ (GH11614)
df.plot
Bug in the groupby plot method when using keyword arguments (GH11805).
groupby
plot
Bug in DataFrame.duplicated and drop_duplicates causing spurious matches when setting keep=False (GH11864)
DataFrame.duplicated
drop_duplicates
keep=False
Bug in .loc result with duplicated key may have Index with incorrect dtype (GH11497)
Bug in pd.rolling_median where memory allocation failed even with sufficient memory (GH11696)
pd.rolling_median
Bug in DataFrame.style with spurious zeros (GH12134)
DataFrame.style
Bug in DataFrame.style with integer columns not starting at 0 (GH12125)
Bug in .style.bar may not rendered properly using specific browser (GH11678)
.style.bar
Bug in rich comparison of Timedelta with a numpy.array of Timedelta that caused an infinite recursion (GH11835)
numpy.array
Bug in DataFrame.round dropping column index name (GH11986)
DataFrame.round
Bug in df.replace while replacing value in mixed dtype Dataframe (GH11698)
df.replace
Dataframe
Bug in Index prevents copying name of passed Index, when a new name is not provided (GH11193)
Bug in read_excel failing to read any non-empty sheets when empty sheets exist and sheetname=None (GH11711)
sheetname=None
Bug in read_excel failing to raise NotImplemented error when keywords parse_dates and date_parser are provided (GH11544)
NotImplemented
parse_dates
date_parser
Bug in read_sql with pymysql connections failing to return chunked data (GH11522)
read_sql
pymysql
Bug in .to_csv ignoring formatting parameters decimal, na_rep, float_format for float indexes (GH11553)
na_rep
float_format
Bug in Int64Index and Float64Index preventing the use of the modulo operator (GH9244)
Bug in MultiIndex.drop for not lexsorted MultiIndexes (GH12078)
MultiIndex.drop
Bug in DataFrame when masking an empty DataFrame (GH11859)
Bug in .plot potentially modifying the colors input when the number of columns didn’t match the number of series provided (GH12039).
.plot
colors
Bug in Series.plot failing when index has a CustomBusinessDay frequency (GH7222).
Series.plot
CustomBusinessDay
Bug in .to_sql for datetime.time values with sqlite fallback (GH8341)
.to_sql
datetime.time
Bug in read_excel failing to read data with one column when squeeze=True (GH12157)
squeeze=True
Bug in read_excel failing to read one empty column (GH12292, GH9002)
Bug in .groupby where a KeyError was not raised for a wrong column if there was only one row in the dataframe (GH11741)
KeyError
Bug in .read_csv with dtype specified on empty data producing an error (GH12048)
.read_csv
Bug in .read_csv where strings like '2E' are treated as valid floats (GH12237)
'2E'
Bug in building pandas with debugging symbols (GH12123)
Removed millisecond property of DatetimeIndex. This would always raise a ValueError (GH12019).
millisecond
Bug in Series constructor with read-only data (GH11502)
Removed pandas._testing.choice(). Should use np.random.choice(), instead. (GH12386)
pandas._testing.choice()
np.random.choice()
Bug in .loc setitem indexer preventing the use of a TZ-aware DatetimeIndex (GH12050)
Bug in .style indexes and MultiIndexes not appearing (GH11655)
.style
Bug in to_msgpack and from_msgpack which did not correctly serialize or deserialize NaT (GH12307).
Bug in .skew and .kurt due to roundoff error for highly similar values (GH11974)
.skew
.kurt
Bug in Timestamp constructor where microsecond resolution was lost if HHMMSS were not separated with ‘:’ (GH10041)
Bug in buffer_rd_bytes src->buffer could be freed more than once if reading failed, causing a segfault (GH12098)
buffer_rd_bytes
Bug in crosstab where arguments with non-overlapping indexes would return a KeyError (GH10291)
crosstab
Bug in DataFrame.apply in which reduction was not being prevented for cases in which dtype was not a numpy dtype (GH12244)
DataFrame.apply
Bug when initializing categorical series with a scalar value. (GH12336)
Bug when specifying a UTC DatetimeIndex by setting utc=True in .to_datetime (GH11934)
utc=True
.to_datetime
Bug when increasing the buffer size of CSV reader in read_csv (GH12494)
Bug when setting columns of a DataFrame with duplicate column names (GH12344)
A total of 101 people contributed patches to this release. People with a “+” by their names contributed a patch for the first time.
ARF +
Alex Alekseyev +
Andrew McPherson +
Andrew Rosenfeld
Andy Hayden
Anthonios Partheniou
Anton I. Sipos
Ben +
Ben North +
Bran Yang +
Chris
Chris Carroux +
Christopher C. Aycock +
Christopher Scanlin +
Cody +
Da Wang +
Daniel Grady +
Dorozhko Anton +
Dr-Irv +
Erik M. Bray +
Evan Wright
Francis T. O’Donovan +
Frank Cleary +
Gianluca Rossi
Graham Jeffries +
Guillaume Horel
Henry Hammond +
Isaac Schwabacher +
Jean-Mathieu Deschenes
Jeff Reback
Joe Jevnik +
John Freeman +
John Fremlin +
Jonas Hoersch +
Joris Van den Bossche
Joris Vankerschaver
Justin Lecher
Justin Lin +
Ka Wo Chen
Keming Zhang +
Kerby Shedden
Kyle +
Marco Farrugia +
MasonGallo +
MattRijk +
Matthew Lurie +
Maximilian Roos
Mayank Asthana +
Mortada Mehyar
Moussa Taifi +
Navreet Gill +
Nicolas Bonnotte
Paul Reiners +
Philip Gura +
Pietro Battiston
RahulHP +
Randy Carnevale
Rinoc Johnson
Rishipuri +
Sangmin Park +
Scott E Lasley
Sereger13 +
Shannon Wang +
Skipper Seabold
Thierry Moisan
Thomas A Caswell
Toby Dylan Hocking +
Tom Augspurger
Travis +
Trent Hauck
Tux1
Varun
Wes McKinney
Will Thompson +
Yoav Ram
Yoong Kang Lim +
Yoshiki Vázquez Baeza
Young Joong Kim +
Younggun Kim
Yuval Langer +
alex argunov +
behzad nouri
boombard +
brian-pantano +
chromy +
daniel +
dgram0 +
gfyoung +
hack-c +
hcontrast +
jfoo +
kaustuv deolal +
llllllllll
ranarag +
rockg
scls19fr
seales +
sinhrks
srib +
surveymedia.ca +
tworec +