Ggthjy

Is there a lava-breathing lizard creature (that could be worshipped by a cult) in 5e?

How much mayhem could I cause as a fish?

How to find the order of a symmetric group S4?

Do authors have to be politically correct in article-writing?

False written accusations not made public - is there law to cover this?

Saint abbreviation

Potential client has a problematic employee I can't work with

Explanation of a regular pattern only occuring for prime numbers

Globe trotting Grandpa. Where is he going next?

How would an AI self awareness kill switch work?

Can you tell from a blurry photo if focus was too close or too far?

Is a new boolean field better than null reference when a value can be meaningfully absent?

Do "fields" always combine by addition?

How to not let the Identify spell spoil everything?

How can the probability of a fumble decrease linearly with more dice?

Square Root Distance from Integers

How do you voice extended chords?

Removing whitespace between consecutive numbers

What makes papers publishable in top-tier journals?

Does diversity provide anything that meritocracy does not?

What will happen if Parliament votes "no" on each of the Brexit-related votes to be held on the 12th, 13th and 14th of March?

Why do we have to make "peinlich" start with a capital letter and also end with -s in this sentence?

Bash script to truncate subject line of incoming email

What is a good reason for every spaceship to carry a weapon on board?

Create an API from EDA or ML outcome?

Pandas: how can I create multi-level columnsHow to explain the outcome of k-means clustering?predict rank from physical measurements with various lengthsCreate a new column based on two columns from two different dataframesCreate new data frames from existing data frame based on unique column valuesCreate top 10 index fund based on >100 stocksCreate price matrix from tidy data without for loopHow to create column for my csv file in pythonHow to create new columns from existing columns with get_dummiesModel to predict based on frequency of occurrence

0

$begingroup$

I have the following sample dataset (the actual dataset is over 10 million records)

     Passenger    Trip

0       Mark     London

1       Mike     Girona

2    Michael      Paris

3        Max     Sydney

4     Martin  Amsterdam

5     Martin  Barcelona

6     Martin  Barcelona

7       Mark     London

8       Mark      Paris

9     Martin   New york

10       Max     Sydney

11       Max      Paris

12       Max     Sydney

...

...

...

And I wanted to get the destination frequently travelled by a passenger !

I was playing around in Jupyter and got the expected data with the following approach

series_px = df_px_dest.groupby('Passenger')['Trip'].apply(lambda x: x.value_counts().head(1))



df_px = series_px.to_frame() 



df_px.index = df_px.index.set_names(['UName', 'DEST'])



df_px.reset_index(inplace=True) 



def getNextPossibleDestByUser(pxname,df=df_px): 

    return df.query('UName==@pxname')['DEST'].to_string(index=False)

While the response is fine. I have few doubts now

1) What's the best way to expose the method (say in this case getNextPossibleDestByUser) as a API (pass customer name as input and get the destination as output) ?

2) Whenever the API is being called , does that mean all the 10 million records gets processed each time ? Are there anyway to optimise that ?

3) Rather than dataframe (pandas) query approach can I consider some ml models or utility functions from say scikit to solve the same problem ?

python scikit-learn pandas

share

asked 8 mins ago

MaddyMaddy

464

$endgroup$

add a comment | 

0

$begingroup$

I have the following sample dataset (the actual dataset is over 10 million records)

     Passenger    Trip

0       Mark     London

1       Mike     Girona

2    Michael      Paris

3        Max     Sydney

4     Martin  Amsterdam

5     Martin  Barcelona

6     Martin  Barcelona

7       Mark     London

8       Mark      Paris

9     Martin   New york

10       Max     Sydney

11       Max      Paris

12       Max     Sydney

...

...

...

And I wanted to get the destination frequently travelled by a passenger !

I was playing around in Jupyter and got the expected data with the following approach

series_px = df_px_dest.groupby('Passenger')['Trip'].apply(lambda x: x.value_counts().head(1))



df_px = series_px.to_frame() 



df_px.index = df_px.index.set_names(['UName', 'DEST'])



df_px.reset_index(inplace=True) 



def getNextPossibleDestByUser(pxname,df=df_px): 

    return df.query('UName==@pxname')['DEST'].to_string(index=False)

While the response is fine. I have few doubts now

1) What's the best way to expose the method (say in this case getNextPossibleDestByUser) as a API (pass customer name as input and get the destination as output) ?

2) Whenever the API is being called , does that mean all the 10 million records gets processed each time ? Are there anyway to optimise that ?

3) Rather than dataframe (pandas) query approach can I consider some ml models or utility functions from say scikit to solve the same problem ?

python scikit-learn pandas

share

asked 8 mins ago

MaddyMaddy

464

$endgroup$

add a comment | 

$begingroup$

I have the following sample dataset (the actual dataset is over 10 million records)

     Passenger    Trip

0       Mark     London

1       Mike     Girona

2    Michael      Paris

3        Max     Sydney

4     Martin  Amsterdam

5     Martin  Barcelona

6     Martin  Barcelona

7       Mark     London

8       Mark      Paris

9     Martin   New york

10       Max     Sydney

11       Max      Paris

12       Max     Sydney

...

...

...

And I wanted to get the destination frequently travelled by a passenger !

I was playing around in Jupyter and got the expected data with the following approach

series_px = df_px_dest.groupby('Passenger')['Trip'].apply(lambda x: x.value_counts().head(1))



df_px = series_px.to_frame() 



df_px.index = df_px.index.set_names(['UName', 'DEST'])



df_px.reset_index(inplace=True) 



def getNextPossibleDestByUser(pxname,df=df_px): 

    return df.query('UName==@pxname')['DEST'].to_string(index=False)

While the response is fine. I have few doubts now

1) What's the best way to expose the method (say in this case getNextPossibleDestByUser) as a API (pass customer name as input and get the destination as output) ?

2) Whenever the API is being called , does that mean all the 10 million records gets processed each time ? Are there anyway to optimise that ?

3) Rather than dataframe (pandas) query approach can I consider some ml models or utility functions from say scikit to solve the same problem ?

python scikit-learn pandas

share

asked 8 mins ago

MaddyMaddy

464

$endgroup$

     Passenger    Trip

0       Mark     London

1       Mike     Girona

2    Michael      Paris

3        Max     Sydney

4     Martin  Amsterdam

5     Martin  Barcelona

6     Martin  Barcelona

7       Mark     London

8       Mark      Paris

9     Martin   New york

10       Max     Sydney

11       Max      Paris

12       Max     Sydney

...

...

...

series_px = df_px_dest.groupby('Passenger')['Trip'].apply(lambda x: x.value_counts().head(1))



df_px = series_px.to_frame() 



df_px.index = df_px.index.set_names(['UName', 'DEST'])



df_px.reset_index(inplace=True) 



def getNextPossibleDestByUser(pxname,df=df_px): 

    return df.query('UName==@pxname')['DEST'].to_string(index=False)

share

asked 8 mins ago

MaddyMaddy

464

share

asked 8 mins ago

MaddyMaddy

464

asked 8 mins ago

MaddyMaddy

464

asked 8 mins ago

MaddyMaddy

464