An interesting feature of this model is that it was trained on Huawei Ascend, instead of NVIDIA chips. In comparison, their last work, GLM-130B, was trained on NVIDIA A100.
That is interesting. I bet we will see a lot more of that in the future because of the NVIDIA export ban. I’ve been looking online to see a performance comparison of the Huawei chips, but I can hardly find any information on them other than marketing material.