This doesn't seem right. There is an error in coding here.

…                
            torch.arange(0, d_model, 2) * -(math.log(10000.0) / d_model)
    )

    # calc sine on even indices
    pe[:, 0::2] = torch.sin(k * div_term)    

    # calc cosine on odd indices   
    pe[:, 1::2] = torch.cos(k * div_term)  

    # add dimension     
    pe = pe.unsqueeze(0)          

    # buffers are saved in state_dic…
Positional Encoding
453
4
Hunter Phillips
Rahul Bhadani
·Follow
Apr 3, 2024
--
This doesn't seem right. There is an error in coding here.
--
--
Written by Rahul Bhadani404 Followers
·165 Following
Asst. Professor, @UAH | @uarizona PhD, ECE | Intelligent Transportation & Quantum Science Researcher | Donation: https://www.buymeacoffee.com/rahulbhadani
No responses yet
Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams