Ggthjy

I have the following dataset

+-----------------------------------+

|  Passenger           |    Trip    |

+-----------------------------------+

| John                 | London     |

| Jack                 | Paris      | 

| Joe                  | Sydney     |

| John                 | London     |

| John                 | London     |

| Jill                 | New york   |

| Jim                  | Sydney     |

| Jack                 | Paris      |

| James                | Sydney     |

+-----------------------------------+

And am trying to use scikit library to predict the likelihood of next possible trip of a passenger based on the frequency ( In this case John => London).
As a novice am unsure on which model / function to use.

Update 2:

If I have over 10 million records , how different should I approach this problem ?

Update 3:
The following code worked for the larger dataset !

series_px = df_px_dest.groupby('Passenger')['Trip'].apply(lambda x: x.value_counts().head(1))



df_px = series_px.to_frame() 



df_px.index = df_px.index.set_names(['UID', 'DEST'])



df_px.reset_index(inplace=True) 



def getNextPossibleDestByUserID(name,df=df_px): 

    return df.query('UID==@name')['DEST'].to_string(index=False)

My next target is to expose that as an API (using Flask maybe) , Will probably raise a new question for that !!

For something like this, you could go with a simpler approach. One idea is to sample randomly among the cities that a given passenger has visited using the amount of times each city has been visited as probabilites.

Here's a way you could do so. I've added a few more examples to the dataframe so that the application is seen more clearly. Say you instead have:

     Passenger    Trip

0       John     London

1       Jack     Girona

2       Jack      Paris

3        Joe     Sydney

4        Joe  Amsterdam

5        Joe  Barcelona

6        Joe  Barcelona

7       John     London

8       John      Paris

9       Jill    Newyork

10       Jim     Sydney

11      Jack      Paris

12     James     Sydney

You could define a function like the folllowing in order to randomly sample from the existing data in the dataframe:

def random_sample(df, name):

    import numpy as np

    # group the dataframe by Passenger and count 

    # the different trips 

    g = df.groupby('Passenger').Trip.value_counts()

    # Make the probabilities add up to 1

    freq = g[name] / g[name].sum()

    # random destination based on 

    # its probabilities

    random_name = np.random.choice(a=freq.index, size=1, 

                     p = freq.values)[0]

    # return likelyhood of next randomly chosen

    # destination and destination

    return freq[random_name], random_name

Usage

Say we want to select a a randomly samples destination for say Joe and also to know which is the likelihood. Considering that the destinations where Joe has been are:

Trip

Barcelona    2

Amsterdam    1

Sydney       1

We could get for example:

for _ in range(5):

    freq, dest = random_sample(df, 'Joe')

    print('Chosen destination {} with a probability of {}'.format(dest, freq))



Chosen destination Sydney with a probability of 0.25

Chosen destination Barcelona with a probability of 0.5

Chosen destination Barcelona with a probability of 0.5

Chosen destination Barcelona with a probability of 0.5

Chosen destination Sydney with a probability of 0.25