You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bluemarlin.apache.org by GitBox <gi...@apache.org> on 2022/02/17 15:54:04 UTC
[GitHub] [incubator-bluemarlin] jimmylao opened a new issue #49: dlpredictor model description
jimmylao opened a new issue #49:
URL: https://github.com/apache/incubator-bluemarlin/issues/49
This is a illustration of the seq2seq model with attention for DLpredictor.
This is the relationship of functions defined in model.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bluemarlin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-bluemarlin] jimmylao commented on issue #49: dlpredictor model description
Posted by GitBox <gi...@apache.org>.
jimmylao commented on issue #49:
URL: https://github.com/apache/incubator-bluemarlin/issues/49#issuecomment-1048165884
@klonikar
1. What is context in the red cells in decoder? Is it used as input to next timestep?
As shown in the 1st figure, the context of the red cells are from the attention layer which will be concatenated with the state output of current cell, then used as input to next cell.
2. Can you explain split of train_window data into train_skip_first, data used for training and final eval? How is the data in the final 10 days of 82 (12+60+10) days used in training? Is it used to update regularization params or weights in backprop?
As shown in the figure above, the data is typical split into configurable 4 parts, i.e. train_skip_first, train_window, validate and back_offset. In your case,
- train_skip_first = 12
- train_window = 60
- predict (validate) = 10
- back_offiset = 0
"Is it used to update regularization params or weights in backprop?" - I don't understand the meaning of this question. The data is used for training a model or leverage trained model for prediction, in training stage, they are used to compute gradient in each batch/epoch which can further be used in backprop.
3. What are the features used at time step of 82 days? How is ts_n in trainready table used as input? I thought that's the output.
As shown in the figure above, time_x is the major input data needed to train the model, it has multiple components. trainready table is the output of preprocessing and input for model training. There's a program called tfrecord_reader.py to convert trainready data format to model training data format.
4. Attention: Explain attention heads = 10, and attention depth = 60
These parameters are used when the "attention" option is turned on. The purpose of attention is to find the correlation among different time step. The attention method used by this model used the attention concept not the implementation, i.e. it uses convolution to model the correlation among different time steps which is faster than typical attention implementation, however, such approximation does not generate better performance. Domain knowledge based attention features, such as: monthly or quarterly correlation, are added in the model to fill the gap and was claimed to work better than using the attention implementation in the model.
5. Can you give mathematical description of the model?
I'll add some math equation later. I suggest you read through the code and run a couple of sample data first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bluemarlin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-bluemarlin] klonikar commented on issue #49: dlpredictor model description
Posted by GitBox <gi...@apache.org>.
klonikar commented on issue #49:
URL: https://github.com/apache/incubator-bluemarlin/issues/49#issuecomment-1046178178
1. What is _context_ in the red cells in decoder? Is it used as input to next timestep?
2. Can you explain split of train_window data into train_skip_first, data used for training and final eval? How is the data in the final 10 days of 82 (12+60+10) days used in training? Is it used to update regularization params or weights in backprop?
3. What are the features used at time step of 82 days? How is ts_n in trainready table used as input? I thought thats the output.
4. Attention: Explain attention heads = 10, and attention depth = 60
5. Can you give mathematical description of the model? Something like:
```
hEncoder(t) = GRU(t-1) + x(t)
x(t) = [ts_n, age, gender...]
```
etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bluemarlin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org