Can I use GAT to modify the network? Will this improve performance?

[haoyu-wangg]

No description provided.

[inoue0426]

You can use the GAT, just replace the function. It might improve the performance.

[haoyu-wangg]

Why do the results vary every time I run the model? I have set the seed, but every time I run the model, the clustering metrics still change, and I found that the loss did not continue to decrease. Does this mean that the purpose of the model is to find a balance between ZINB loss and MSE loss? I am a beginner, and I would be grateful if you could answer this question.Thanks!

[inoue0426]

So basically, deep learning is not stable even though you set the seed. https://arxiv.org/abs/2202.02326

Depends on the randomness, dropout, initial weight, etc..., the loos is not continuously decreasing. This is common in deep learning.

[haoyu-wangg]

Thank you. If I change to GAT, do I need to change the hyperparameters and epochs? I find that the early stop strategy is not suitable for this task but why? My mentor is asking me to do this task, and you are my savior, thanks again.

[inoue0426]

Need to change all parameters. Loss decreasing is not directory related to the clustering metrics. It depends on the task.

[haoyu-wangg]

I noticed that the paper describes the parameter settings as follows: "We set the parameters α at 0.05 and epochs at 100 during this process. Model optimization was performed using the Adam algorithm [14] with a learning rate 5e-4. These parameter values and the number of layers were defined by leveraging Optuna [1] for parameter tuning." Could you explain how Optuna can be specifically applied to this task? What is the optimization objective function, and which parameters need to be adjusted using Optuna? If you still have the process of hyperparameter search, could you please contact me at [email protected]? I would greatly appreciate it.