Posts tagged with "NPU"

Reading the Dispatch API gives you the list of things to ask a chip vendor before you commit, and explains why TOPS is the least useful number on the datasheet. Third in a series on LiteRT and on-device inference.

讀懂 Dispatch API，你就拿到了押注一顆晶片之前該問廠商的清單，也會明白為什麼 TOPS 是規格表上最沒用的數字。LiteRT 端側推理系列第三篇。

Where the cuts land decides whether acceleration is a win or a loss, and JIT carries a consequence the docs state plainly but that's easy to skim past. Second in a series on LiteRT and on-device inference.

切在哪裡決定了加速到底是賺還是賠，而 JIT 有一個文件寫得很清楚、卻很容易掃過去的代價。LiteRT 端側推理系列第二篇。

Cracking open a real .tflite, and reading grouped-query attention straight off the tensor shapes. First in a series on LiteRT and on-device inference, starting with what a model file physically is.

把真實的 .tflite 剖開來看裡面裝什麼，順便從 tensor shape 讀出模型用了 GQA。LiteRT 端側推理系列第一篇，從模型檔案的實體結構講起。