Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: fix DataFrame(data=[None, 1], dtype='timedelta64[ns]') raising ValueError #60081

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

BUG: fix DataFrame(data=[None, 1], dtype='timedelta64[ns]') raising ValueError #60081

wants to merge 1 commit into from
+9 −1

Conversation

yuanx749
Copy link
Contributor

@yuanx749
Copy link
Contributor Author

The problem comes from the branch elif is_float_dtype(data.dtype): in function sequence_to_td64ns.

pandas/pandas/core/arrays/timedeltas.py

Line 1114 in 68d9dca

data = cast_from_unit_vectorized(data, unit or "ns")

data here is a 2D array([[nan], [ 1.]]).
But cast_from_unit_vectorized only accepts 1D array as input, thus producing the "Buffer has wrong number of dimensions" error (see issue), if I understand correctly the code below:

pandas/pandas/_libs/tslibs/conversion.pyx

Lines 133 to 147 in 68d9dca

cdef:
ndarray[int64_t] base, out
ndarray[float64_t] frac
tuple shape = (<object>values).shape
out = np.empty(shape, dtype="i8")
base = np.empty(shape, dtype="i8")
frac = np.empty(shape, dtype="f8")
for i in range(len(values)):
if is_nan(values[i]):
base[i] = NPY_NAT
else:
base[i] = <int64_t>values[i]
frac[i] = values[i] - base[i]

I notice that other branches for other dtypes in sequence_to_td64ns can work with 2D array data, so come up with this quick fix.

@yuanx749 yuanx749 marked this pull request as ready for review October 22, 2024 04:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@yuanx749