Ggthjy

Why are the books in the Game of Thrones citadel library shelved spine inwards?

Separate environment for personal and development use under macOS

What is the difference between rolling more dice versus fewer dice?

Is there a lava-breathing lizard creature (that could be worshipped by a cult) in 5e?

Does diversity provide anything that meritocracy does not?

How do I append a character to the end of every line in an excel cell?

Short story where statues have their heads replaced by those of carved insect heads

Play Zip, Zap, Zop

I have trouble understanding this fallacy: "If A, then B. Therefore if not-B, then not-A."

A starship is travelling at 0.9c and collides with a small rock. Will it leave a clean hole through, or will more happen?

In Linux what happens if 1000 files in a directory are moved to another location while another 300 files were added to the source directory?

Why did Democrats in the Senate oppose the Born-Alive Abortion Survivors Protection Act (2019 S.130)?

What will happen if Parliament votes "no" on each of the Brexit-related votes to be held on the 12th, 13th and 14th of March?

After checking in online, how do I know whether I need to go show my passport at airport check-in?

Do authors have to be politically correct in article-writing?

What is the difference between "...", '...', $'...', and $"..." quotes?

Why does photorec keep finding files after I have filled the disk free space as root?

How does one write from a minority culture? A question on cultural references

Am I a Rude Number?

Building an exterior wall within an exterior wall for insulation

Why did Luke use his left hand to shoot?

Why do we have to make "peinlich" start with a capital letter and also end with -s in this sentence?

Cat is tipping over bed-side lamps during the night

What will happen if I transfer large sums of money into my bank account from a pre-paid debit card or gift card?

Evaluating value functions in RL

What is the Q function and what is the V function in reinforcement learning?Reward dependent on (state, action) versus (state, action, successor state)Cannot see what the “notation abuse” is, mentioned by author of bookHow is that possible that a reward function depends both on the next state and an action from current state?state-action-reward-new state: confusion of termsRegarding the Value function in Proximal Policy OptimizationNeed help in deriving Policy Evaluation (Prediction)What is the optimal value of a Markov Decision process with Single actions at each state?How is Importance-Sampling Used in Off-Policy Monte Carlo Prediction?RL Advantage function why A = Q-V instead of A=V-Q?

0

$begingroup$

I'm working my way through the book Reinforcement Learning by Richar S. Sutton and Andrew G. Barto and I am stuck on the following question.

The value of a state depends on the the values of the actions possible in that state and on how likely each action is to be taken under the current policy.
We can think of this in terms of a small backup diagram rooted at the state and considering each possible action:

Give the equation corresponding to this intuition and diagram for the value at the root node
, in terms of the value at the expected leaf node,
, given
. This expectation depends on the policy,
. Then give a second equation in which the expected value is written out explicitly in terms of
such that no expected value notation appears in the equation.

I should mention that...

...

Where...

= Probability of taking action a from state s

= Given any state s and a, the probability of each next state s'

= Expected reward given any state s, next state s, and action a

How can I re-evaluate this value function in the way that is asked?

reinforcement-learning markov-process monte-carlo

share

asked 3 mins ago

BolboaBolboa

1011

New contributor

Bolboa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

0

$begingroup$

I'm working my way through the book Reinforcement Learning by Richar S. Sutton and Andrew G. Barto and I am stuck on the following question.

The value of a state depends on the the values of the actions possible in that state and on how likely each action is to be taken under the current policy.
We can think of this in terms of a small backup diagram rooted at the state and considering each possible action:

Give the equation corresponding to this intuition and diagram for the value at the root node
, in terms of the value at the expected leaf node,
, given
. This expectation depends on the policy,
. Then give a second equation in which the expected value is written out explicitly in terms of
such that no expected value notation appears in the equation.

I should mention that...

...

Where...

= Probability of taking action a from state s

= Given any state s and a, the probability of each next state s'

= Expected reward given any state s, next state s, and action a

How can I re-evaluate this value function in the way that is asked?

reinforcement-learning markov-process monte-carlo

share

asked 3 mins ago

BolboaBolboa

1011

New contributor

Bolboa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I'm working my way through the book Reinforcement Learning by Richar S. Sutton and Andrew G. Barto and I am stuck on the following question.

The value of a state depends on the the values of the actions possible in that state and on how likely each action is to be taken under the current policy.
We can think of this in terms of a small backup diagram rooted at the state and considering each possible action:

Give the equation corresponding to this intuition and diagram for the value at the root node
, in terms of the value at the expected leaf node,
, given
. This expectation depends on the policy,
. Then give a second equation in which the expected value is written out explicitly in terms of
such that no expected value notation appears in the equation.

I should mention that...

...

Where...

= Probability of taking action a from state s

= Given any state s and a, the probability of each next state s'

= Expected reward given any state s, next state s, and action a

How can I re-evaluate this value function in the way that is asked?

reinforcement-learning markov-process monte-carlo

share

asked 3 mins ago

BolboaBolboa

1011

New contributor

Bolboa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

share

asked 3 mins ago

BolboaBolboa

1011

New contributor

Bolboa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share

asked 3 mins ago

BolboaBolboa

1011

New contributor

Bolboa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 3 mins ago

BolboaBolboa

1011

New contributor

Bolboa is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 3 mins ago

BolboaBolboa

1011