Recently a small company in the People's Republic of China released a GPT-LLM model set called Deep Seek. This seems to have surprised some in the US tech oligarchy to the tune of about a trillion dollars.
This is presented by some commentators as ‘little China’ being nimbler than ‘big America’. But this is not strictly true, and Frank Herbert anticipated the point in Appendix I of his novel Dune.
Big America is in reality a bandwagon started by OpenAI and Nvidia around a specific architecture epitomised by Chat-GPT.
This demo caught the common imagination and was backed by Elon Harkonnen, Eric Corrino, Mark Richese, Sam Atreides and others. They organised investment in the $100 Bn range. Eric even said at a closed meeting at Stanford that the technology would do what his “15,000 programmers would not do”.
Little China was one company within the PRC headed by one Liang Wenfeng, which claims to have spent about $6m on electricity for their demo, using older export-grade chips from the USA.
How many firms and universities in the PRC failed to produce anything? We do not know. We will never know. Deep Seek found a sweet spot in the landscape of possibilities. We do not know what combination of skill, money, inspiration, technology, and luck led them to this. But they got there.
What did Frank Herbert say about finding success? In Appendix 1 of Dune he wrote:
Kynes knew that highly organised research is guaranteed to produce nothing new. He set up small-unit experiments with regular interchange of data for a swift Tansley effect.
Sir Arthur George Tansley FRS was an English botanist and a pioneer in the science of ecology. There is a lot of stuff on him online. He organised committees. One term - 'ecosystem' - associated with him was actually from a colleague. He was an inspiration, but not perhaps a Great Man.
Small teams with a lot of lateral information sharing versus a highly organised structure run from the centre. I shall not labour the point more.
Trillion-dollar pivots can come about by quite humble people noting that an Emperor is actually naked. Maybe 9999 people at that time miss that point. It might take only one though.
There are a lot of possible configurations in the space of GPT-LLM. Multiple ones may be valuable. Herbert used the example of ecology in his Appendix to appear profound. Ignoring this might cost someone a trillion dollars.
Peter Chambers is a Lib Dem Member in Hampshire.
Glossary
GPT – generative pre-trained transformer, something that generates an output by transforming an input using pre-trained (fixed) weights, mathematically. The inputs and outputs are digital data.
Chat-GPT – a demonstration GPT made by the firm OpenAI. It obtained viral PR fame.
FRS – Fellow of the Royal Society
LLM – large language model, a model that uses a language of digital data tokens, which are strings of bits. Make it a large one. The meaning and significance of the tokens exists in the senses and minds of humans, natural intelligences that evolved on the Third Planet.
PRC – the People’s Republic of China ("the Mainland one").
No comments:
Post a Comment