Pandas Dataframe Reshaping/stacking Of Multiple Value Variables Into Seperate Columns

Hi I'm trying to reshape a data frame in a certain way. this is the data frame I have, des1 des2 des3 interval1 interval2 interval3 value aaa a b c ##1

Solution 1:

This might be a shorter approach:

[72]:

df.columns = pd.MultiIndex.from_tuples(map(lambda x: (x[:-1], x), df.columns))
In [73]:

print pd.DataFrame({key:df[key].stack().values for key inset(df.columns.get_level_values(0))},
                   index = df['des'].stack().index.get_level_values(0))
      des interval
value             
aaa     a      ##1
aaa     b      ##2
aaa     c      ##3
bbb     d      ##4
bbb     e      ##5
bbb     f      ##6
ccc     g      ##7
ccc     h      ##8
ccc     i      ##9

Or preserve the 1,2,3 info:

[73]:

df.columns = pd.MultiIndex.from_tuples(map(lambda x: (x[:-1], x[-1]), df.columns))
Keys = set(df.columns.get_level_values(0))
df2  = pd.concat([df[key].stack() for key in Keys], axis=1)
df2.columns = Keys
print df2
        des interval
value               
aaa   1   a      ##12   b      ##23   c      ##3
bbb   1   d      ##42   e      ##53   f      ##6
ccc   1   g      ##72   h      ##83   i      ##9

Solution 2:

This is just a .melt, docs are here

In[33]: pd.melt(df.reset_index(),
                 id_vars=['values'],
                 value_vars=['interval1','interval2','interval3'])
Out[33]: 
  valuesvariablevalue0aaainterval1   ##11bbbinterval1   ##42cccinterval1   ##73aaainterval2   ##24bbbinterval2   ##55cccinterval2   ##86aaainterval3   ##37bbbinterval3   ##68cccinterval3   ##9

Solution 3:

I think the solution provided by CT Zhu is very genius. But you also can reshape this step by step (maybe this is the common way).

 d = {'des1' : ['', 'a', 'd', 'g'],
     'des2' : ['', 'b', 'e', 'h'],
     'des3' : ['', 'c', 'f', 'i'],
     'interval1' : ['', '##1', '##4', '##7'],
     'interval2' : ['', '##2', '##5', '##6'],
     'interval3' : ['', '##3', '##6', '##9']}

df = pd.DataFrame(d, index=['value', 'aaa', 'bbb', 'ccc'], 
                  columns=['des1', 'des2', 'des3', 'interval1', 'interval2', 'interval3'])

nd = {'des' : [''] + df.iloc[1, 0:3].tolist() + df.iloc[2, 0:3].tolist() + df.iloc[3, 0:3].tolist(),
      'interval' : ['']+ df.iloc[1, 3:6].tolist() + df.iloc[2, 3:6].tolist() + df.iloc[3, 3:6].tolist()}

ndf = pd.DataFrame(nd, index=['value', 'aaa', 'aaa', 'aaa', 'bbb', 'bbb', 'bbb', 'ccc', 'ccc', 'ccc'], columns=['des', 'interval'])

Solution 4:

This type of reshaping can be done conveniently with pandas.wide_to_long:

import io
import pandas as pd  # v 1.2.3

data = '''
value des1 des2 des3 interval1 interval2 interval3  
aaa  a  b  c ##1 ##2 ##3
bbb  d  e  f ##4 ##5 ##6
ccc  g  h  i ##7 ##8 ##9
'''
df = pd.read_csv(io.StringIO(data), index_col=0, delim_whitespace=True)

pd.wide_to_long(df.reset_index(), stubnames=['des', 'interval'],
                i='value', j='var_id').droplevel(1).sort_index()