Why Is Speculative Decoding Fast?

4 comments

• ArashEdalat 37 minutes ago
  we have action chunking in robotics for the same reason so VLAs can analyze faster and come to actions, the latency and inference are both slow so many steps need to come together for this to happen correctly
• ape4 23 minutes ago
  systemd-resolved can send your query to multiple servers and use the one that returns first.
• ArashEdalat 36 minutes ago
  has anyone tried different approaches to this?
  [-]
  • guipsp 23 minutes ago
    ...yes? Can you put more effort into your question?
• wheeeeel 24 minutes ago
  [flagged]