NewsLab
Oct 27 15:37 UTC

The Exponential Time Hypothesis Is False [pdf] (github.com)

731 points|by vok||261 comments|Read full story on github.com
See also: Announcing three new discoveries in mathematics, computer science, and physics - https://news.ycombinator.com/item?id=50251344

Comments (261)

120 shown|More comments
  1. 1. lisper||context
    This seems huge, like not quite p=np but in the same league. Am I missing something ?
  2. 2. adgjlsfhk1||context
    no. It's huge.
  3. 3. krackers||context
    [deleted, originally wrote the hypothesis being true would imply NP=EXPTIME which is wrong, since there can be problems in EXPTIME which aren't in NP.

    And the result that it's now false apparently doesn't imply NP!=EXPTIME either since it seems not strongly subexponential enough to offset the expansion in variables from reductions. See the other comments for a much better explanation.]

  4. 4. luc4||context
    I mean the result that k-SAT cannot be solved in 2^{o(n)} clearly would imply P!=NP, but how does ETH imply NP = EXP? And how does \neg ETH imply NP != EXP?
  5. 5. eru||context
    > Instead we now know that NP is a strict subset of EXP (there are problems in NP that can be solved in subexponential time).

    It's a bit more complicated than that, if you want to be careful. Sorting is in both NP and EXP and can be solved in subexponential time.

    I think what they proved is that no problem in NP takes exponential time to solve. (Because you can always reduce them to SAT in polynomial time, and then solve SAT in subexponential time.)

    EDIT: apparently the runtime bounds they gave here don't survive the usual polynomial reductions, thus they haven't proven that NP != EXP, it seems.

  6. 6. odyssey7||context
    It’s not that it isn’t quite p=np; it actually isn’t p=np.

    Still, an impressive accomplishment.

  7. 7. lynndotpy||context
    If I am understanding this correctly (and I am still reading this!) and if this is true, this means NP is sub-exponential. Sub-exponential does not mean P=NP. But yes, this is huge if true.

    This means hard problems aren't as hard as we thought, and it's still possible P=NP. If it turns out to be true P=NP, we're a little closer to that. (I hope P!=NP!)

    Every paper which assumes ETH is true might now have their conclusions be disproven.

  8. 8. cochne||context
    It doesn't imply NP is sub-exponential, only that 3SAT is. Conversions to other problems can still end up exponential.
  9. 9. wbl||context
    No. NP is always poly time reduceable to 3SAT
  10. 10. cochne||context
    See this from the paper: "Note that runtime bounds of the form 2^o(n) do not transfer under general polynomial-time reductions (2^n^o(1) would be needed for this), so our result does not imply that general NP problems admit subexponential time algorithms."
  11. 11. adgjlsfhk1||context
    OTOH, it would be surprising if this sort of technique didn't apply to other NP problems.
  12. 12. jibalt||context
    It's not clear what you're trying to say or the relevance. The fact remains that "NP is always poly time reduceable to 3SAT" and "3SAT is sub-exponential" does not imply that NP is sub-exponential.
  13. 13. adgjlsfhk1||context
    right, but anything linearly reducable to kSAT is trivially sub-exponential, and a discovery like this means that linear reductions are a whole lot more important than they used to be.

    Also for NP problems that can't linear reduce to kSAT just having the knowledge that SAT is sub-exponential (and the tools), makes it a lot more likely that we see a whole lot more problems become exponential.

  14. 14. calf||context
    Yeah that would be interesting, like could there be different NP languages being sub-exponential but for entirely different reasons.
  15. 15. dataflow||context
    Did you reduce in the wrong direction or am I misunderstanding what you're trying to say?
  16. 16. mswphd||context
    practically, a poly-time reduction can increase teh size of hte problem instance. So maybe you transform a 3-color instance (graph on n vertices) into a n^2 sized SAT instance. You can then solve this in time 2^{n^2 / sqrt{log{n^2}}}, but this is not sub-exponential.
  17. 17. jibalt||context
    Your "No" is wrong. As he said, "Conversions to other problems can still end up exponential", so (NP is poly time reduceable to 3SAT + 3SAT is sub-exponential) does not imply that NP is sub-exponential -- the cost of the reduction may be enough to push the conjunction from sub-exponential to exponential.

    Note https://news.ycombinator.com/item?id=50253213 which explains it better.

  18. 18. LPisGood||context
    Subexponential(poly(n)) may be exponential in n
  19. 19. lynndotpy||context
    Yeah, I was wrong. Page two too, I commented too soon.

    NP is not yet proven to be subexponential but still, ETH disproven is huge

  20. 20. jaggederest||context
    This defeats a few hypotheses that were widely relied on.

    strong exponential time hypothesis, and the titular exponential time hypothesis

    It's not saying that these problems are easy, but it does demonstrate that they are much easier than previously believed. A decade ago if a human had published this kind of thing, and importantly if it held up, they would be almost certainly a strong candidate for some sort of award(s).

  21. 21. thatoneengineer||context
    Assuming P != NP (and that this NP != EXP proof holds up), what's the next most "natural" candidate for how fast NP must grow? This paper appears to show Omega(2^(n/sqrt(log(n))) with a couple additional fudge factors which would be a very disappointing answer.
  22. 22. zeroonetwothree||context
    Definitely puts some weight on the P=NP side of the scale!
  23. 23. jibalt||context
    This is not a proof that NP != EXP. NP is poly time reduceable to 3SAT + 3SAT is sub-exponential does not imply that NP is sub-exponential -- the cost of the reduction may be enough to push the conjunction from sub-exponential to exponential.

    Note https://news.ycombinator.com/item?id=50253213 which explains it better.

  24. 24. xigoi||context
    Even if all NP problems could be solved in sub-exponential time, it would not imply that NP ≠ EXP. It seems that many people here are confused about what EXP means – it’s the set of all problems that can be solved in at most exponential time, including those in P.
  25. 25. xigoi||context
    This is not a proof of NP ≠ EXP. NP ≠ EXP would mean that there is a problem that can be solved in exponential time, but not in non-deterministic polynomial time. That’s completely different from what they have shown.
  26. 26. SkiFire13||context
    Not sure about how "natural" Omega(2^(n/sqrt(log(n))) is, but a huge intermediate result would be proving that QP = NP or QP != NP.
  27. 27. kakapo5672||context
    My (very amateur) interpretation is that this weakens confidence in P!=NP. Would like to hear from some experts in this area, if this is so or not.
  28. 28. caughtinthought||context
    It basically just says that NP is not the same as EXP
  29. 29. jibalt||context
    This is not a proof that NP != EXP. NP is poly time reduceable to 3SAT + 3SAT is sub-exponential does not imply that NP is sub-exponential -- the cost of the reduction may be enough to push the conjunction from sub-exponential to exponential.

    Note https://news.ycombinator.com/item?id=50253213 which explains it better.

  30. 30. pcstl||context
    This proves that EXP != NP, which was a stronger claim than P != NP.

    In a way it shows that NP is perhaps less distant from P than imagined. Whether it can be _reduced_ to P in some gigabrain way, or whether that can be shown to be impossible remains to be seen.

    It is definitely progress towards an answer to P ?= NP.

  31. 31. jibalt||context
    This is not a proof that NP != EXP. NP is poly time reduceable to 3SAT + 3SAT is sub-exponential does not imply that NP is sub-exponential -- the cost of the reduction may be enough to push the conjunction from sub-exponential to exponential.

    Note https://news.ycombinator.com/item?id=50253213 which explains it better.

  32. 32. mudita||context
    Scott Aaronson comments on his blog "Given this news, I’ll up my personal probability of P=NP from 3% to 5%." (https://scottaaronson.blog/?p=10239#comment-2042428)
  33. 33. zeroonetwothree||context
    I’m not as smart as Scott but for me it’s more like 10% -> 20%. Do we have a prediction market for this?
  34. 34. kibwen||context
    Predictions markets are terrible mechanisms for this, because they require imposing an arbitrary time limit at which point the prediction will pay out. IOW, even if you religiously believed P=NP, but you think it would take at least two years to prove it, you would vote against the prediction if the deadline were one year.
  35. 35. internetter||context
    Why must prediction markets have a predefined end date?
  36. 36. jibalt||context
    Without it, capital would be trapped indefinitely.
  37. 37. howunfortunate||context
    This is not necessarily true, you could have contracts rather than money down
  38. 38. throw-qqqqq||context
    Sure, but in most (all?) financial settings, this would still count as exposure.

    It doesn’t matter if I bet $N on something using cash or margin, my exposure is still $N.

    Since you can’t have unlimited exposure, this would still lock up capital, until the bet can be resolved.

  39. 39. FeepingCreature||context
    Well, so? Then the prediction market would also price expected resolution time.
  40. 40. Tadpole9181||context
    How would it work otherwise...?
  41. 41. internetter||context
    Anybody can exit their contract by selling their contract to an interested party
  42. 42. jibalt||context
    It seems highly irrational to bet that your evaluation is more likely than Aaronson's.
  43. 43. lostmsu||context
    Not under the circumstances IMO
  44. 44. IsTom||context
    There's a chain of classes P, NP(aka Sigma_1), Sigma_2, .., Sigma_n, ... PSPACE, EXPTIME. What we know for sure is that P < EXPTIME, others are unknown (well, maybe this paper adds something to this, time will tell).

    It's conceivable that each of them is smaller than the next, but also that e.g. P < NP = all the sigmas < PSPACE < EXPTIME or many other possibilities. I think the general consensus is that it's more likely that there are more inequalities than not, but the fact is that we can't even show that P is less than PSPACE so far, which would be extraordinarily surprising if it were true.

  45. 45. amelius||context
    Subexponential time still can mean much larger than polynomial time. There's a huge space between the two.
  46. 46. mlmonkey||context
    Can someone ELI5 ETH for those of us who slept through theoretical CS classes? Asking for a friend...
  47. 47. caughtinthought||context
    3SAT can't be solved in sub-exponential time

    Fwiw never thought it was a well-named conjecture

  48. 48. pcstl||context
    *Can be solved

    This proves that EXP != NP

  49. 49. caughtinthought||context
    I'm stating the conjecture as requested
  50. 50. pcstl||context
    I think you misunderstood, I believe they were requesting a layperson-friendly explanation of the conjecture and not a statement of it.
  51. 51. caughtinthought||context
    Which is a separate issue than the one you commented originally
  52. 52. moishe_syzslak||context
    Whenever someone asks for an ELI5, there's always got to be somebody who further confuses the issue.
  53. 53. jibalt||context
    How is "This proves that EXP != NP", which is false, a layperson-friendly explanation of the actual proof that 3SAT is sub-exponential?

    Note https://news.ycombinator.com/item?id=50253213

  54. 54. jibalt||context
    This is not a proof that NP != EXP. NP is poly time reduceable to 3SAT + 3SAT is sub-exponential does not imply that NP is sub-exponential -- the cost of the reduction may be enough to push the conjunction from sub-exponential to exponential.

    Note https://news.ycombinator.com/item?id=50253213 which explains it better.

  55. 55. ehershey||context
    I like this https://www.reddit.com/r/explainlikeimfive/comments/33fmrq/e...

      In computer science, the satisfiability problem is the task of determining whether a boolean problem can be satisfied, that is, made to be true, by some combination of inputs. For example, the expression: A AND NOT B is satisfied whenever A is true and B is false, while the expression: A AND NOT A cannot be satisfied.
      A naive approach to satisfiablilty would be to check every possible input and see if the expression evaluates to true. 
      However, the number of combinations that must be checked is 2n, where n is the number of variables in the expression. This is what is meant by exponential time.
      The exponential time hypothesis conjectures that you cannot do better, that is, the number of operations needed to solve satisfiability will always be exponential in the size of the problem, no matter what algorithm is chosen.
      If this hypothesis is true, it would imply that P≠NP, because satisfiability has been shown to be in NP, and in order for it to be in P there would have to exist an algorithm that solves it in polynomial time.
  56. 56. gberger||context
    2n should read as 2^n
  57. 57. newsy-combi||context
    Unless OP already solved p=np...
  58. 58. pvab3||context
    that threw me, thanks
  59. 59. nullc||context
    > the number of operations needed to solve satisfiability will always be exponential in the size of the problem, no matter what algorithm is chosen

    A common complexity theory misunderstanding is that statements like this applies to all or even most instances-- and it leads to objectively bad engineering of the form "oh look this is actually 3sat, it's unsolvable".

    (Great fodder for a midwit meme: idiot and genius say "my heuristics go brrr!"-- midwit says "No no no! NP problems can't be efficiently solved!")

    Any specific instance may be efficiently solvable (and perhaps all instances coming from your particular problem set -- a subset of all possible 3sats).

    So I would add to that paragraph "at least for some 3sat instances".

  60. 60. marshray||context
    My 2nd grader once brought home a homework sheet with some simple problems involving combinations of coins and I was like "this is unsolvable!"
  61. 61. pcstl||context
    The exponential time hypothesis is essentially that NP-complete problems' best solutions are all exponential.

    NP-complete problems are essentially those where you need to make some kind of choice over a superlinearly-growing set of options where each option can be solved in polynomial time. Like, for example, finding a route between cities on a map under a certain total distance. You might need to go over all choices of routes.

    The ETH essentially assumes that the best solution (in terms of complexity, i.e. how it grows) you can get for these kinds of problems is exponential. OpenAI just proved that this is false, you can have solutions that are below exponential complexity (but still above polynomial - if anyone finds a polynomial solution then we'd have proven that P=NP)

  62. 62. eru||context
    > NP-complete problems are essentially those where you need to make some kind of choice over a superlinearly-growing set of options where each option can be solved in linear time.

    Seems like a weird way to describe NP, and I'm not even sure it's true?

    NP is the class of decision problems where if the answer is YES and someone slips you a hint, you can solve them quickly (or quickly figure out that the hint was useless and report that). Where quickly means in polynomial time.

    There's nothing about linear time in the classic definition.

  63. 63. pcstl||context
    Sorry, you're correct. I meant "polynomial" rather than linear.

    NP = nondeterministically polynomial. i.e. if you could evaluate all choices in parallel (or just "know" the right choice), then you'd get a polynomial runtime. That's what I was trying to informally hint at, but I ended up using the wrong term. I'll edit my original post.

    That's equivalent to your description - the "hint" is knowing which of the options to choose.

    The reason why superlinearity comes into play is that if you had a linearly growing amount of solutions to test then you'd just have a sum of polynomials, which is itself polynomial.

  64. 64. eru||context
    Ok, I read your edited comment. Looks a lot better know.

    One nitpick: I think your set of options needs to grow a lot faster than just superlinearly, I suspect it needs to grew superpolynomially?

    However otherwise what you describe is essentially true of all problems in NP, not just the NP hard problems:

    Sorting is (trivially) in NP, and you can say that to sort you need to pick one out of n! many permutations (which certainly grows superlinearly and even superpolynomially).

    NP hard problems have the distinction that if you solve one of them efficiently, you can solve all of them efficiently, because you can reduce them to each other. (Basically because NP hard problems allow you to sort-of describe a computer, and reducing them to each other is equivalent to writing an interpreter or compiler.)

  65. 65. SyzygyRhythm||context
    There's a set of problems that are (it is conjectured) easy to verify but hard to solve. One of them, called 3SAT, looks for values (a, b, c, ...) such that an expression like ((a or b or c) and (b or not-c or e) and ...) evaluates to true. And it turns out that all of these supposedly hard problems can be converted to 3SAT, so if you find a solution to one you find a solution to all.

    The Exponential Time Hypothesis in effect says that 3SAT can only be solved by brute force--trying every possible combination until you find the right one. If your 3SAT expression has 100 variables, then this will take about 2^100 tries (exponential time).

    What they found was an algorithm faster than brute force. 3SAT is still potentially one of these hard problems, but finding anything faster than brute force is a major advance.

  66. 66. dnautics||context
    the language around complexity domains is really confusing.

    P means "this problem can be solved in polynomial # of steps relative to the problem size"

    NP means "we don't know if the problem can be solved in polynomial # of steps relative to problem size, but we know we can check if the result is correct in a polynomial # of steps relative to the problem size"

    !ETH: it was widely suspected that the flagship problem in NP, 3SAT, requires an exponential # of steps relative to problem size. We now know that 3SAT requires AT MOST something more than polynomial but strictly less than exponential to solve.

    "P = NP" means "if you can check it in polynomial time, there's some way to solve it in polynomial time". "P != NP" means "there are some problems checkable in polynomial time for which there is no polynomial way to obtain the result"

    Today's new proof makes "P = NP" "feel more likely", because we know that the flagship problem in NP is not exponential.

  67. 67. tfpgh||context
    This isn't quite true, problems in NP can still require exponential time to solve.

    Solving an arbitrary NP problem looks like first reducing that problem to SAT and then solving the equivalent SAT problem. We know (via the Cook–Levin theorem) that any NP problem can be reduced to SAT in polynomial time, and we now know that solving SAT is subexponential in terms of the size of the SAT problem, but it's possible for the conversion to SAT to grow the problem enough that it's exponential in terms of the original problem.

  68. 68. dnautics||context
    thanks for the correction. Corrected.
  69. 69. jibalt||context
    But even if every problem in NP is sub-exponential (which, as tfpgh notes, this result doesn't prove), how does that make it "more likely" that every such problem is solvable in a polynomial # of steps? I feel like it's a bit like claiming that, because Alex Honnold has shown that a human can free solo El Capitan in less than 4 hours, that makes it "more likely" that I could. To me, his success does not budge the latter probability at all.

    It seems to me that the logic only goes the other way ... if ETH had been proven then it would follow that P != NP. With !ETH we really don't know any more about P ?= NP then we did before. Reasoning that P = NP is "more likely" strikes me as a form of a fallacy of affirming the consequent.

  70. 70. whattheheckheck||context
    Is this really as interpetable math is fundamentally. Like wtf is this accessibility? who is this for? Do top mathematicians read this like a chapter book? Absolutely ridiculous they have all this compute and creative resources and cant make something more intuitive to help people understand it more natively
  71. 71. NetMageSCW||context
    Sometimes hard things are hard.
  72. 72. atlex2||context
    Actually, no - this is what controllability is all about. Humans have to understand it.
  73. 73. elzbardico||context
    Humans can understand it. Not all of them, because to understanding it, there's a lot of other things you need to understand before.
  74. 74. vecter||context
    It's a math paper written for mathematicians.

    It's not possible for someone who's not a specialist in the field to understand it (unless there are more elementary proofs, of which likely none exist).

  75. 75. eru||context
    Even mathematicians take a long time to read math books and articles, especially if they go into the details. They don't read math like you would read a novel.
  76. 76. xeonmc||context
    > They don't read math like you would read a novel.

    siting on a recliner making an occasional sensible chuckle while flipping through the pages?

  77. 77. paxy||context
    It's sometimes okay to look at something, go "this isn't for me", and move on. You don't need to have an opinion on everything.
  78. 78. whattheheckheck||context
    Same to you bud
  79. 79. serf||context
    I agree with you insofar that the primary motivator for these press releases is public relations and investor confidence, so it should be made as minimally hard to understand as possible just by sheer virtue that it exposes oAI to more cash thrown at them.

    I totally disagree with you on the precept that all difficult to understand things need be needlessly dumbed-down/watered-down until palatable for a mass (average) audience. Requiring 'math50' before 'trig101' isn't gate-keeping in any moral sense, it's an effort to save everyones' time involved when the effort is futile.

    Similarly I don't demand that haute couture fashion companies explain the style to me until 'I get it' ; I just take it for granted that i'm not going to be able to understand the entirety of the world in one lifetime.

  80. 80. eru||context
    > I agree with you insofar that the primary motivator for these press releases is public relations and investor confidence, so it should be made as minimally hard to understand as possible just by sheer virtue that it exposes oAI to more cash thrown at them.

    Why? The investors can just ask Claude or Gemini whether the proof holds up.

  81. 81. ks2048||context
    > Absolutely ridiculous they have all this compute and creative resources and cant make something more intuitive to help people understand it more natively

    OpenAI has that too - open their website and ask.

  82. 82. whattheheckheck||context
    And get some hallucinated slop infotainment?
  83. 83. baggy_trough||context
    [flagged]
  84. 84. freecodeio||context
    math major here, I can't read the paper for shit
  85. 85. eru||context
    That problem seems independent of the 'stochastic parrot' accusation?
  86. 86. freecodeio||context
    A parrot might squeak the theory of everything and it won't matter to any of us cause nobody speaks parrot.
  87. 87. eru||context
    We can record a lot of parrot speech and train LLMs against it and eventually build a translator.

    They are making good progress on similar research programmes for many cetaceans.

    In the case of our stochastic parrots it's even easier, since they already speak something like human language. And for the math proofs it's even easier, because once they are formalised into something like Lean the humans only need to understand the statement of the theorem, and a mechanical computer program can check the proof.

  88. 88. freecodeio||context
    Yes we can just train llms against it and it's all solved. Just a few more trillion dollars and a bunch of deforestation for datacenters bro.
  89. 89. CaptWorld||context
    Ironically yeah.. if it can solve more complex problems across domains like biology, econ, finance then all the deforestation is worth it
  90. 90. freecodeio||context
    I would take a living, breathing forest over math papers that nobody knows what to do with. Hell I would take a leaf with an ant over it.
  91. 91. moishe_syzslak||context
    Not a math major here. But is this paper way too long? So long that it raises the possibility that OpenAI's staff mathematicians passed through AI's raw output as-is?
  92. 92. elzbardico||context
    It is just that we never considered how advanced stochastic parrots could be so powerful when dealing with basically the whole library of acquired human knowledge and guided by experts to generate.

    It is not a demerit, actually I think that it is a very elegant outcome how many problems could be solved by generation and this probably show us how language is a far more important thing than we ever thought it was.

  93. 93. criemen||context
    Wow. Complexity class separations are really really rare.

    Can an expert in sat solving comment if there's any immediate applications of the method from the paper? Or is SAT solving in general pretty far away from worst-case times, and this is a theoretical algorithm which would be hard to apply in practice.

  94. 94. infamouscow||context
    The data you put into a SAT solver has substantial effect on it's performance in practice. Real world inputs tend to be much nicer than pathological ones, so I don't think there's much practical application, but I also haven't read the paper yet.
  95. 95. kingstnap||context
    I also haven't read ths paper but coding agents absolutely are a part of modern SAT solver coding challenges.
  96. 96. bsubs||context
    SAT-solving expert here. No immediate applications (the results in the paper are "galactic"). The main question is whether some ideas here will snowball into other ideas, one of which could eventually be very practical.
  97. 97. ashkankiani||context
    What do you mean by "galactic"?
  98. 98. yyyk||context
    Galactic algorithms have such huge constants they figuratively need a galaxy-sized computer and/or galaxy-sized problem, to have any advantage over the normal solutions. Imagine a generic sorting algorithm in O(n) that had a constant of 10^92 **. Technically superior but in practice unusable.

    ** Yes, I know that's proven to be impossible.

  99. 99. MichaelDickens||context
    There are realistic situations where galactic O(n) sorting algorithms exist, right? 32-bit integers can be sorted in O(n) using 2^32 words of memory, using bucket sort[1].

    If objects have a maximum size of 4 GB (= 2^35 bits), then arbitrary objects can be sorted in O(n) using 2^2^35 bits of memory.

    [1] https://en.wikipedia.org/wiki/Bucket_sort

  100. 100. yyyk||context
    Right, that's why I wrote "a generic sorting algorithm". Your case adds the integer limit which allows it to avoid O(nlogn) minimum.
  101. 101. adrianN||context
    You can use Radix sort https://en.wikipedia.org/wiki/Radix_sort to lower the memory requirement for limited bit-width integers. But there are more practical o(n log n) methods for sorting integers.
  102. 102. adgjlsfhk1||context
    Well optimized radix sorts are actually pretty fast (assuming you have the proper fallbacks to quicksort/insertion sort to deal with medium/small sizes)
  103. 103. kite42||context
  104. 104. gradus_ad||context
    Do we know they are galactic?
  105. 105. zahlman||context
    > A galactic algorithm is an algorithm with state-of-the-art theoretical (asymptotic) performance, but which is not used due to practical constraints.

    Yes, I think we can pretty safely say that the algorithm in the paper is not used due to practical constraints. This is a subjective judgment, but pretty clear from the paper that the algorithms aren't practical to apply.

  106. 106. sebmellen||context
    This is a complete non-sequitur, but I checked out your web page. I love the Curious George image. My daughter has been deeply into the Curious George episodes for the past couple of months, and I don't think I've seen him show up in many other places besides kids' shows :D
  107. 107. chr15m||context
    Such a good show. My kids grew up on it!
  108. 108. angry_octet||context
    I think of Curious George Goes to a Chocolate Factory when I fire up Claude Code.
  109. 109. strommec||context
    It's surprising that Exponential Time Hypothesis would be false. Quoting a paper from 2022[1]:

    > We prove that the time complexity of self-attention is necessarily quadratic in the input length, unless the Strong Exponential Time Hypothesis (SETH) is false[...]

    If OpenAI's proof is correct, does that mean self-attention can be implemented in sub-quadratic time? I.e. we could eventually have all the most advanced LLMs running on cheap hardware? To put it more playfully, is ETH being false actually bearish for Nvidia?

    [1]: https://arxiv.org/abs/2209.04881

  110. 110. rokob||context
    No, SETH implies quadratic time does not mean !SETH implies !quadratic time.
  111. 111. dnautics||context
    but you're telling me.... there's a chance
  112. 112. Xirdus||context
    This but unironically.
  113. 113. programjames||context
    put a number on your probabilities. there's a chance you have an aneurysm in the next few minutes and fail to see my comment. if so, oh well, i tried.
  114. 114. caughtinthought||context
    SETH is a much stronger claim than ETH
  115. 115. margalabargala||context
    Yes, it's one strong stronger.
  116. 116. jibalt||context
    Not relevant, because !ETH => !SETH

    What is relevant is that the logic goes the other way: SETH implies quadratic, !SETH implies nothing.

  117. 117. varyherb||context
    Right, so a refutation of ETH would also refute SETH.
  118. 118. refulgentis||context
    It's playful!

    But it's also not even wrong, in the Pauli sense.

    That makes me wonder if "I see your CS proof, how does this affect LLMs / nvidia stonk price???" being immediately self-described as "playful" ensures criticism looks cranky.

  119. 119. Joker_vD||context
    "I see your particle physics paper (on sustainability of neutron-induced chain fission reaction in uranium), how does this affect the world politics???"

    Turns out, it affected the global politics tremendously.

  120. 120. _ink_||context
    If AI really scales indefinitely is an increase in efficiency even bearish for Nvidia? Wouldn't they still build more and more compute as fast as they can, because these things get smarter and better with more compute?