python - pandas read frame and subsequent type manipulation issue -


hello rookie pandas , have particular behavior want know reason for

sqlnew = 'select fund_no,start_date,end_date,cash,bond,small_cap,large_cap,international odsact.act_src_fund_mapping;' actualfundmapping = psql.read_frame(sqlnew,cnxn) 

'everyworks fine until above'

actualfundmapping.dtypes::  fund_no           object start_date        object end_date          object cash             float64 bond             float64 small_cap        float64 large_cap        float64 international    float64 

'so since want stuff datetime.datetime(2013,1,1) in actualfundmapping['start_date'] try changing dtype below

actualfundmapping['start_date'] = pd.to_datetime(str(actualfundmapping['start_date'])) actualfundmapping['end_date'] = pd.to_datetime(str(actualfundmapping['end_date'])) actualfundmapping['fund_no'] = actualfundmapping['fund_no'].astype(np.int64) 

'but existence tests come false

datetime.datetime(2012,1,1) in actualfundmapping['start_date'] false 

"that entry exists in sql table , have verified it

where as

str(datetime.datetime(2012,1,1)) in str(actualfundmapping['start_date']) true 

'the same issue repeats following

np.int64(1000) in actualfundmapping['fund_no'] false 

i guess not understanding representation pandas use internally. appreciated.

*******update*******

upon further advice made start_date column of sql table index column of dataframe.

actualfundmapping = psql.read_frame(sqlnew,cnxn,'start_date')

actualfundmapping.index
[2012-01-01 00:00:00, ..., 2012-01-01 00:00:00]
length: 895, freq: none, timezone: none

datetime.datetime(2012,1,1) in actualfundmapping false 

datetime.datetime(2012,1,1) in actualfundmapping.index

false

np.datetime64(datetime.datetime(2012,1,1)) in actualfundmapping.index

false

by doing str(actualfundmapping['start_date']), converting entire column single string. believe pandas stores strings dtype object (to avoid truncation issues arise when using numpy string dtypes). try using pd.to_datetime(actualfundmapping['start_date']). if raises error, there may mixed-type data in column, , have more detective work figure out individual values causing problems.

as unutbu suggests, need sure check membership in series values, not index.


Comments

Popular posts from this blog

python - mat is not a numerical tuple : openCV error -

c# - MSAA finds controls UI Automation doesn't -

wordpress - .htaccess: RewriteRule: bad flag delimiters -