Artificial basic intelligence (AGI) is a kind of expert system (AI) that matches or goes beyond human cognitive capabilities across a large range of cognitive jobs. This contrasts with narrow AI, which is limited to specific tasks. [1] Artificial superintelligence (ASI), on the other hand, describes AGI that considerably goes beyond human cognitive abilities. AGI is thought about among the definitions of strong AI.

Creating AGI is a primary objective of AI research and of companies such as OpenAI [2] and Meta. [3] A 2020 study determined 72 active AGI research study and advancement jobs throughout 37 nations. [4]

The timeline for achieving AGI remains a topic of continuous dispute amongst scientists and specialists. As of 2023, some argue that it may be possible in years or decades; others maintain it may take a century or longer; a minority believe it may never ever be attained; and another minority claims that it is already here. [5] [6] Notable AI researcher Geoffrey Hinton has expressed issues about the fast development towards AGI, recommending it might be accomplished quicker than many anticipate. [7]

There is dispute on the precise meaning of AGI and relating to whether modern-day large language models (LLMs) such as GPT-4 are early kinds of AGI. [8] AGI is a common subject in sci-fi and futures research studies. [9] [10]

Contention exists over whether AGI represents an existential risk. [11] [12] [13] Many experts on AI have actually stated that alleviating the danger of human extinction posed by AGI needs to be a global priority. [14] [15] Others find the advancement of AGI to be too remote to present such a risk. [16] [17]

Terminology

AGI is also called strong AI, [18] [19] complete AI, [20] human-level AI, [5] human-level smart AI, or general intelligent action. [21]

Some scholastic sources schedule the term "strong AI" for computer programs that experience life or awareness. [a] In contrast, weak AI (or narrow AI) is able to fix one specific issue however lacks general cognitive capabilities. [22] [19] Some scholastic sources use "weak AI" to refer more broadly to any programs that neither experience awareness nor have a mind in the very same sense as humans. [a]

Related ideas consist of synthetic superintelligence and transformative AI. An artificial superintelligence (ASI) is a theoretical kind of AGI that is far more normally intelligent than humans, [23] while the idea of transformative AI connects to AI having a big effect on society, for instance, similar to the farming or industrial transformation. [24]

A structure for categorizing AGI in levels was proposed in 2023 by Google DeepMind scientists. They define five levels of AGI: emerging, proficient, expert, virtuoso, and superhuman. For instance, a proficient AGI is specified as an AI that surpasses 50% of skilled grownups in a wide variety of non-physical tasks, and a superhuman AGI (i.e. a synthetic superintelligence) is likewise specified however with a threshold of 100%. They consider big language designs like ChatGPT or LLaMA 2 to be circumstances of emerging AGI. [25]

Characteristics

Various popular meanings of intelligence have been proposed. One of the leading proposals is the Turing test. However, there are other well-known definitions, and some researchers disagree with the more popular approaches. [b]

Intelligence characteristics

Researchers generally hold that intelligence is needed to do all of the following: [27]

factor, use strategy, fix puzzles, and make judgments under unpredictability
represent knowledge, consisting of typical sense understanding
plan
learn
- communicate in natural language
- if necessary, integrate these abilities in conclusion of any provided objective


Many interdisciplinary techniques (e.g. cognitive science, computational intelligence, and choice making) think about extra characteristics such as imagination (the ability to form novel psychological images and principles) [28] and autonomy. [29]

Computer-based systems that show numerous of these abilities exist (e.g. see computational imagination, automated reasoning, decision assistance system, robot, evolutionary computation, intelligent agent). There is debate about whether modern-day AI systems possess them to an appropriate degree.

Physical characteristics

Other abilities are considered desirable in smart systems, as they may impact intelligence or aid in its expression. These include: [30]

- the ability to sense (e.g. see, hear, etc), and
- the ability to act (e.g. move and control objects, change area to explore, etc).


This consists of the ability to find and react to risk. [31]

Although the ability to sense (e.g. see, hear, etc) and the ability to act (e.g. move and control objects, modification area to explore, and so on) can be desirable for some smart systems, [30] these physical capabilities are not strictly required for an entity to certify as AGI-particularly under the thesis that big language designs (LLMs) may already be or end up being AGI. Even from a less optimistic viewpoint on LLMs, there is no company requirement for an AGI to have a human-like type; being a silicon-based computational system suffices, offered it can process input (language) from the external world in place of human senses. This analysis aligns with the understanding that AGI has never been proscribed a particular physical embodiment and therefore does not demand a capacity for mobility or conventional "eyes and ears". [32]

Tests for human-level AGI

Several tests implied to confirm human-level AGI have been thought about, pl.velo.wiki consisting of: [33] [34]

The concept of the test is that the maker has to try and pretend to be a guy, by responding to questions put to it, and it will just pass if the pretence is reasonably persuading. A significant part of a jury, who must not be expert about devices, must be taken in by the pretence. [37]

AI-complete issues

An issue is informally called "AI-complete" or "AI-hard" if it is believed that in order to solve it, one would require to execute AGI, because the solution is beyond the abilities of a purpose-specific algorithm. [47]

There are lots of problems that have been conjectured to require general intelligence to resolve along with human beings. Examples include computer vision, natural language understanding, and handling unforeseen situations while fixing any real-world issue. [48] Even a specific task like translation needs a maker to read and write in both languages, follow the author's argument (reason), comprehend the context (understanding), and consistently replicate the author's initial intent (social intelligence). All of these issues require to be fixed concurrently in order to reach human-level machine performance.

However, a lot of these tasks can now be carried out by modern large language designs. According to Stanford University's 2024 AI index, AI has reached human-level efficiency on many benchmarks for checking out understanding and visual thinking. [49]

History

Classical AI

Modern AI research study began in the mid-1950s. [50] The very first generation of AI researchers were convinced that artificial general intelligence was possible which it would exist in just a few decades. [51] AI pioneer Herbert A. Simon wrote in 1965: "machines will be capable, within twenty years, of doing any work a man can do." [52]

Their forecasts were the inspiration for Stanley Kubrick and Arthur C. Clarke's character HAL 9000, who embodied what AI scientists thought they could create by the year 2001. AI pioneer Marvin Minsky was a specialist [53] on the job of making HAL 9000 as sensible as possible according to the consensus predictions of the time. He stated in 1967, "Within a generation ... the problem of creating 'artificial intelligence' will considerably be solved". [54]

Several classical AI tasks, such as Doug Lenat's Cyc project (that started in 1984), and Allen Newell's Soar project, were directed at AGI.

However, in the early 1970s, it became obvious that scientists had actually grossly undervalued the difficulty of the job. Funding companies became skeptical of AGI and put researchers under increasing pressure to produce helpful "used AI". [c] In the early 1980s, Japan's Fifth Generation Computer Project restored interest in AGI, setting out a ten-year timeline that included AGI goals like "continue a table talk". [58] In reaction to this and the success of specialist systems, both market and government pumped money into the field. [56] [59] However, self-confidence in AI spectacularly collapsed in the late 1980s, and the goals of the Fifth Generation Computer Project were never ever fulfilled. [60] For the second time in 20 years, AI researchers who forecasted the impending accomplishment of AGI had actually been mistaken. By the 1990s, AI researchers had a credibility for making vain guarantees. They became hesitant to make predictions at all [d] and prevented mention of "human level" synthetic intelligence for worry of being labeled "wild-eyed dreamer [s]. [62]

Narrow AI research

In the 1990s and early 21st century, mainstream AI accomplished business success and academic respectability by focusing on particular sub-problems where AI can produce verifiable outcomes and business applications, such as speech acknowledgment and recommendation algorithms. [63] These "applied AI" systems are now used thoroughly throughout the innovation market, and research in this vein is heavily funded in both academic community and market. Since 2018 [upgrade], advancement in this field was considered an emerging trend, and a mature phase was expected to be reached in more than ten years. [64]

At the turn of the century, many traditional AI scientists [65] hoped that strong AI could be established by combining programs that solve different sub-problems. Hans Moravec wrote in 1988:

I am positive that this bottom-up route to synthetic intelligence will one day fulfill the conventional top-down route more than half method, all set to supply the real-world skills and the commonsense understanding that has been so frustratingly elusive in thinking programs. Fully intelligent machines will result when the metaphorical golden spike is driven joining the two efforts. [65]

However, even at the time, this was disputed. For example, Stevan Harnad of Princeton University concluded his 1990 paper on the symbol grounding hypothesis by mentioning:

The expectation has actually frequently been voiced that "top-down" (symbolic) approaches to modeling cognition will somehow meet "bottom-up" (sensory) approaches somewhere in between. If the grounding factors to consider in this paper are valid, then this expectation is hopelessly modular and there is really only one feasible route from sense to signs: from the ground up. A free-floating symbolic level like the software application level of a computer system will never be reached by this route (or vice versa) - nor is it clear why we should even try to reach such a level, given that it looks as if arriving would just amount to uprooting our symbols from their intrinsic significances (consequently merely lowering ourselves to the practical equivalent of a programmable computer). [66]

Modern synthetic general intelligence research

The term "synthetic basic intelligence" was utilized as early as 1997, by Mark Gubrud [67] in a discussion of the ramifications of fully automated military production and operations. A mathematical formalism of AGI was proposed by Marcus Hutter in 2000. Named AIXI, the proposed AGI representative maximises "the capability to satisfy goals in a wide variety of environments". [68] This kind of AGI, defined by the ability to increase a mathematical meaning of intelligence rather than display human-like behaviour, [69] was also called universal expert system. [70]

The term AGI was re-introduced and promoted by Shane Legg and Ben Goertzel around 2002. [71] AGI research study activity in 2006 was described by Pei Wang and Ben Goertzel [72] as "producing publications and initial results". The very first summer season school in AGI was organized in Xiamen, China in 2009 [73] by the Xiamen university's Artificial Brain Laboratory and OpenCog. The very first university course was given in 2010 [74] and 2011 [75] at Plovdiv University, Bulgaria by Todor Arnaudov. MIT provided a course on AGI in 2018, organized by Lex Fridman and including a number of visitor lecturers.

As of 2023 [upgrade], a little number of computer system researchers are active in AGI research study, and lots of contribute to a series of AGI conferences. However, significantly more scientists are interested in open-ended knowing, [76] [77] which is the concept of allowing AI to continuously learn and innovate like human beings do.

Feasibility

As of 2023, the development and possible achievement of AGI remains a topic of intense debate within the AI neighborhood. While standard consensus held that AGI was a remote objective, recent improvements have actually led some scientists and market figures to declare that early forms of AGI might already exist. [78] AI leader Herbert A. Simon speculated in 1965 that "devices will be capable, within twenty years, of doing any work a man can do". This prediction failed to come true. Microsoft co-founder Paul Allen believed that such intelligence is unlikely in the 21st century due to the fact that it would require "unforeseeable and fundamentally unforeseeable advancements" and a "scientifically deep understanding of cognition". [79] Writing in The Guardian, roboticist Alan Winfield declared the gulf in between contemporary computing and human-level expert system is as broad as the gulf in between existing area flight and practical faster-than-light spaceflight. [80]

A further difficulty is the absence of clarity in defining what intelligence entails. Does it require awareness? Must it display the capability to set objectives as well as pursue them? Is it purely a matter of scale such that if design sizes increase adequately, intelligence will emerge? Are facilities such as preparation, reasoning, and causal understanding needed? Does intelligence require clearly reproducing the brain and its specific faculties? Does it need feelings? [81]

Most AI scientists believe strong AI can be accomplished in the future, but some thinkers, like Hubert Dreyfus and Roger Penrose, deny the possibility of accomplishing strong AI. [82] [83] John McCarthy is among those who believe human-level AI will be accomplished, but that today level of development is such that a date can not accurately be forecasted. [84] AI specialists' views on the expediency of AGI wax and subside. Four polls conducted in 2012 and 2013 suggested that the typical quote among professionals for when they would be 50% confident AGI would show up was 2040 to 2050, depending upon the poll, with the mean being 2081. Of the experts, 16.5% responded to with "never" when asked the very same question however with a 90% confidence rather. [85] [86] Further existing AGI progress considerations can be discovered above Tests for validating human-level AGI.

A report by Stuart Armstrong and Kaj Sotala of the Machine Intelligence Research Institute discovered that "over [a] 60-year amount of time there is a strong predisposition towards anticipating the arrival of human-level AI as between 15 and 25 years from the time the prediction was made". They evaluated 95 forecasts made in between 1950 and 2012 on when human-level AI will come about. [87]

In 2023, Microsoft researchers published a comprehensive assessment of GPT-4. They concluded: "Given the breadth and depth of GPT-4's abilities, we think that it might reasonably be seen as an early (yet still incomplete) version of an artificial general intelligence (AGI) system." [88] Another study in 2023 reported that GPT-4 outperforms 99% of people on the Torrance tests of imaginative thinking. [89] [90]

Blaise Agüera y Arcas and Peter Norvig composed in 2023 that a significant level of general intelligence has actually already been attained with frontier designs. They wrote that reluctance to this view comes from 4 primary factors: a "healthy skepticism about metrics for AGI", an "ideological commitment to alternative AI theories or strategies", a "dedication to human (or biological) exceptionalism", or a "issue about the financial implications of AGI". [91]

2023 likewise marked the introduction of big multimodal designs (large language designs capable of processing or producing multiple methods such as text, audio, and images). [92]

In 2024, OpenAI launched o1-preview, the first of a series of designs that "spend more time believing before they respond". According to Mira Murati, this ability to think before reacting represents a new, additional paradigm. It enhances model outputs by spending more computing power when creating the response, whereas the model scaling paradigm improves outputs by increasing the model size, training data and training calculate power. [93] [94]

An OpenAI worker, Vahid Kazemi, claimed in 2024 that the company had attained AGI, mentioning, "In my opinion, we have actually already attained AGI and it's a lot more clear with O1." Kazemi clarified that while the AI is not yet "better than any human at any task", it is "better than a lot of people at the majority of jobs." He also dealt with criticisms that large language designs (LLMs) simply follow predefined patterns, comparing their knowing procedure to the scientific technique of observing, hypothesizing, and validating. These statements have actually sparked debate, as they rely on a broad and non-traditional meaning of AGI-traditionally comprehended as AI that matches human intelligence throughout all domains. Critics argue that, while OpenAI's designs demonstrate remarkable adaptability, they might not completely meet this requirement. Notably, Kazemi's remarks came quickly after OpenAI got rid of "AGI" from the terms of its collaboration with Microsoft, triggering speculation about the company's tactical intents. [95]

Timescales

Progress in expert system has actually historically gone through periods of rapid progress separated by durations when progress appeared to stop. [82] Ending each hiatus were fundamental advances in hardware, software or both to create space for further progress. [82] [98] [99] For instance, the hardware available in the twentieth century was not sufficient to carry out deep learning, which requires large numbers of GPU-enabled CPUs. [100]

In the intro to his 2006 book, [101] Goertzel says that quotes of the time needed before a really flexible AGI is built vary from ten years to over a century. Since 2007 [upgrade], the consensus in the AGI research community seemed to be that the timeline gone over by Ray Kurzweil in 2005 in The Singularity is Near [102] (i.e. between 2015 and 2045) was possible. [103] Mainstream AI researchers have provided a wide variety of opinions on whether progress will be this quick. A 2012 meta-analysis of 95 such viewpoints found a predisposition towards predicting that the onset of AGI would happen within 16-26 years for modern and historic predictions alike. That paper has been slammed for how it categorized viewpoints as expert or non-expert. [104]

In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton established a neural network called AlexNet, which won the ImageNet competition with a top-5 test error rate of 15.3%, considerably better than the second-best entry's rate of 26.3% (the standard approach used a weighted amount of scores from various pre-defined classifiers). [105] AlexNet was concerned as the initial ground-breaker of the existing deep knowing wave. [105]

In 2017, researchers Feng Liu, Yong Shi, and Ying Liu conducted intelligence tests on publicly readily available and easily accessible weak AI such as Google AI, Apple's Siri, and others. At the maximum, these AIs reached an IQ worth of about 47, which corresponds approximately to a six-year-old kid in first grade. An adult comes to about 100 on average. Similar tests were carried out in 2014, with the IQ rating reaching a maximum value of 27. [106] [107]

In 2020, OpenAI established GPT-3, a language model efficient in carrying out lots of diverse tasks without particular training. According to Gary Grossman in a VentureBeat post, while there is agreement that GPT-3 is not an example of AGI, it is considered by some to be too advanced to be classified as a narrow AI system. [108]

In the exact same year, Jason Rohrer utilized his GPT-3 account to establish a chatbot, and provided a chatbot-developing platform called "Project December". OpenAI requested for modifications to the chatbot to abide by their safety guidelines; Rohrer disconnected Project December from the GPT-3 API. [109]

In 2022, DeepMind established Gato, a "general-purpose" system capable of carrying out more than 600 various jobs. [110]

In 2023, Microsoft Research released a study on an early version of OpenAI's GPT-4, competing that it showed more basic intelligence than previous AI models and demonstrated human-level performance in tasks spanning several domains, such as mathematics, coding, and law. This research stimulated an argument on whether GPT-4 could be considered an early, incomplete variation of artificial general intelligence, emphasizing the need for further exploration and assessment of such systems. [111]

In 2023, the AI scientist Geoffrey Hinton specified that: [112]

The concept that this things might in fact get smarter than individuals - a couple of people thought that, [...] But the majority of people thought it was method off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer believe that.

In May 2023, Demis Hassabis similarly stated that "The progress in the last couple of years has been pretty incredible", and that he sees no reason that it would slow down, expecting AGI within a decade and even a couple of years. [113] In March 2024, Nvidia's CEO, Jensen Huang, specified his expectation that within five years, AI would be capable of passing any test at least in addition to people. [114] In June 2024, the AI scientist Leopold Aschenbrenner, a former OpenAI worker, approximated AGI by 2027 to be "strikingly possible". [115]

Whole brain emulation

While the advancement of transformer designs like in ChatGPT is considered the most appealing course to AGI, [116] [117] whole brain emulation can work as an alternative technique. With whole brain simulation, a brain design is built by scanning and mapping a biological brain in information, and then copying and imitating it on a computer system or another computational device. The simulation model need to be sufficiently faithful to the initial, so that it behaves in virtually the very same method as the initial brain. [118] Whole brain emulation is a kind of brain simulation that is talked about in computational neuroscience and neuroinformatics, and for medical research purposes. It has actually been gone over in expert system research study [103] as a method to strong AI. Neuroimaging technologies that could deliver the needed detailed understanding are improving quickly, and futurist Ray Kurzweil in the book The Singularity Is Near [102] anticipates that a map of enough quality will become readily available on a comparable timescale to the computing power required to replicate it.

Early approximates

For low-level brain simulation, a very effective cluster of computer systems or GPUs would be required, provided the massive quantity of synapses within the human brain. Each of the 1011 (one hundred billion) nerve cells has on average 7,000 synaptic connections (synapses) to other neurons. The brain of a three-year-old kid has about 1015 synapses (1 quadrillion). This number declines with age, stabilizing by their adult years. Estimates differ for an adult, varying from 1014 to 5 × 1014 synapses (100 to 500 trillion). [120] A quote of the brain's processing power, based on a basic switch design for nerve cell activity, is around 1014 (100 trillion) synaptic updates per second (SUPS). [121]

In 1997, Kurzweil looked at numerous price quotes for the hardware required to equal the human brain and embraced a figure of 1016 calculations per 2nd (cps). [e] (For contrast, if a "calculation" was equivalent to one "floating-point operation" - a procedure utilized to rate existing supercomputers - then 1016 "calculations" would be comparable to 10 petaFLOPS, accomplished in 2011, while 1018 was accomplished in 2022.) He utilized this figure to forecast the necessary hardware would be offered sometime between 2015 and 2025, if the exponential development in computer power at the time of writing continued.

Current research study

The Human Brain Project, an EU-funded effort active from 2013 to 2023, has developed an especially comprehensive and publicly accessible atlas of the human brain. [124] In 2023, scientists from Duke University carried out a high-resolution scan of a mouse brain.

Criticisms of simulation-based approaches

The artificial nerve cell design presumed by Kurzweil and used in lots of present artificial neural network executions is basic compared to biological nerve cells. A brain simulation would likely need to catch the detailed cellular behaviour of biological neurons, currently comprehended only in broad summary. The overhead introduced by full modeling of the biological, chemical, and physical details of neural behaviour (particularly on a molecular scale) would need computational powers several orders of magnitude larger than Kurzweil's price quote. In addition, the price quotes do not account for glial cells, which are known to contribute in cognitive procedures. [125]

A fundamental criticism of the simulated brain method stems from embodied cognition theory which asserts that human personification is an important aspect of human intelligence and is necessary to ground significance. [126] [127] If this theory is appropriate, any totally functional brain design will need to incorporate more than simply the nerve cells (e.g., a robotic body). Goertzel [103] proposes virtual embodiment (like in metaverses like Second Life) as an option, but it is unknown whether this would be adequate.

Philosophical point of view

"Strong AI" as specified in philosophy

In 1980, theorist John Searle coined the term "strong AI" as part of his Chinese room argument. [128] He proposed a distinction in between two hypotheses about expert system: [f]

Strong AI hypothesis: An expert system system can have "a mind" and "consciousness".
Weak AI hypothesis: An expert system system can (just) act like it thinks and has a mind and awareness.


The first one he called "strong" due to the fact that it makes a more powerful statement: it presumes something special has occurred to the device that exceeds those abilities that we can check. The behaviour of a "weak AI" machine would be specifically identical to a "strong AI" machine, however the latter would also have subjective mindful experience. This use is also common in scholastic AI research study and books. [129]

In contrast to Searle and traditional AI, some futurists such as Ray Kurzweil utilize the term "strong AI" to mean "human level artificial basic intelligence". [102] This is not the very same as Searle's strong AI, unless it is assumed that awareness is needed for human-level AGI. Academic thinkers such as Searle do not think that holds true, and to most expert system scientists the question is out-of-scope. [130]

Mainstream AI is most thinking about how a program behaves. [131] According to Russell and Norvig, "as long as the program works, they don't care if you call it real or a simulation." [130] If the program can act as if it has a mind, then there is no need to understand if it in fact has mind - undoubtedly, there would be no other way to inform. For AI research study, Searle's "weak AI hypothesis" is comparable to the statement "artificial general intelligence is possible". Thus, according to Russell and Norvig, "most AI researchers take the weak AI hypothesis for approved, and do not care about the strong AI hypothesis." [130] Thus, for academic AI research study, "Strong AI" and "AGI" are 2 various things.

Consciousness

Consciousness can have different meanings, and some elements play significant roles in science fiction and the ethics of artificial intelligence:

Sentience (or "remarkable consciousness"): The ability to "feel" perceptions or emotions subjectively, instead of the capability to factor about understandings. Some philosophers, such as David Chalmers, utilize the term "consciousness" to refer specifically to extraordinary consciousness, which is approximately equivalent to life. [132] Determining why and how subjective experience arises is called the hard problem of awareness. [133] Thomas Nagel explained in 1974 that it "seems like" something to be mindful. If we are not conscious, then it doesn't feel like anything. Nagel utilizes the example of a bat: we can smartly ask "what does it feel like to be a bat?" However, we are unlikely to ask "what does it seem like to be a toaster?" Nagel concludes that a bat seems conscious (i.e., has awareness) however a toaster does not. [134] In 2022, a Google engineer declared that the company's AI chatbot, LaMDA, had attained life, though this claim was extensively challenged by other professionals. [135]

Self-awareness: To have conscious awareness of oneself as a different person, particularly to be consciously familiar with one's own thoughts. This is opposed to merely being the "topic of one's believed"-an operating system or debugger has the ability to be "knowledgeable about itself" (that is, to represent itself in the exact same way it represents whatever else)-but this is not what people normally suggest when they utilize the term "self-awareness". [g]

These traits have a moral measurement. AI sentience would generate issues of well-being and legal defense, similarly to animals. [136] Other aspects of consciousness associated to cognitive capabilities are likewise appropriate to the idea of AI rights. [137] Finding out how to integrate advanced AI with existing legal and social frameworks is an emergent problem. [138]

Benefits

AGI could have a wide range of applications. If oriented towards such objectives, AGI might assist alleviate different issues on the planet such as cravings, poverty and health issue. [139]

AGI might enhance productivity and efficiency in a lot of tasks. For instance, in public health, AGI might speed up medical research study, notably versus cancer. [140] It might look after the elderly, [141] and equalize access to quick, premium medical diagnostics. It might use fun, inexpensive and tailored education. [141] The need to work to subsist might become obsolete if the wealth produced is properly rearranged. [141] [142] This likewise raises the concern of the location of human beings in a significantly automated society.

AGI might also assist to make rational choices, and to anticipate and prevent disasters. It could also assist to reap the advantages of potentially disastrous innovations such as nanotechnology or climate engineering, while preventing the associated dangers. [143] If an AGI's primary goal is to avoid existential catastrophes such as human extinction (which could be difficult if the Vulnerable World Hypothesis turns out to be real), [144] it might take steps to drastically reduce the risks [143] while minimizing the impact of these measures on our lifestyle.

Risks

Existential dangers

AGI may represent several kinds of existential threat, which are threats that threaten "the early termination of Earth-originating intelligent life or the long-term and drastic damage of its potential for preferable future advancement". [145] The danger of human extinction from AGI has actually been the topic of lots of arguments, however there is also the possibility that the development of AGI would lead to a permanently flawed future. Notably, it could be utilized to spread and maintain the set of values of whoever establishes it. If mankind still has moral blind spots similar to slavery in the past, AGI might irreversibly entrench it, avoiding ethical progress. [146] Furthermore, AGI could help with mass surveillance and indoctrination, which might be used to produce a steady repressive worldwide totalitarian routine. [147] [148] There is likewise a threat for the makers themselves. If makers that are sentient or otherwise worthy of moral consideration are mass produced in the future, engaging in a civilizational path that forever overlooks their well-being and interests could be an existential catastrophe. [149] [150] Considering just how much AGI might enhance humankind's future and help in reducing other existential threats, Toby Ord calls these existential dangers "an argument for continuing with due caution", not for "deserting AI". [147]

Risk of loss of control and human extinction

The thesis that AI postures an existential risk for people, which this risk requires more attention, is controversial but has been endorsed in 2023 by many public figures, AI scientists and CEOs of AI business such as Elon Musk, Bill Gates, Geoffrey Hinton, Yoshua Bengio, Demis Hassabis and Sam Altman. [151] [152]

In 2014, Stephen Hawking criticized widespread indifference:

So, facing possible futures of enormous advantages and risks, the experts are certainly doing everything possible to guarantee the very best result, right? Wrong. If a superior alien civilisation sent us a message stating, 'We'll arrive in a few years,' would we simply respond, 'OK, call us when you get here-we'll leave the lights on?' Probably not-but this is basically what is occurring with AI. [153]

The possible fate of humanity has in some cases been compared to the fate of gorillas threatened by human activities. The contrast mentions that higher intelligence allowed mankind to dominate gorillas, which are now vulnerable in manner ins which they might not have actually prepared for. As an outcome, the gorilla has ended up being a threatened species, not out of malice, however merely as a security damage from human activities. [154]

The skeptic Yann LeCun considers that AGIs will have no desire to control mankind and that we should beware not to anthropomorphize them and interpret their intents as we would for human beings. He stated that individuals will not be "smart adequate to design super-intelligent makers, yet ridiculously stupid to the point of offering it moronic goals with no safeguards". [155] On the other side, the principle of instrumental convergence recommends that nearly whatever their goals, smart agents will have reasons to attempt to survive and obtain more power as intermediary steps to accomplishing these goals. Which this does not need having feelings. [156]

Many scholars who are worried about existential threat supporter for more research into fixing the "control issue" to address the concern: what types of safeguards, algorithms, or architectures can developers carry out to increase the possibility that their recursively-improving AI would continue to act in a friendly, rather than devastating, manner after it reaches superintelligence? [157] [158] Solving the control problem is made complex by the AI arms race (which could lead to a race to the bottom of security preventative measures in order to release products before competitors), [159] and the usage of AI in weapon systems. [160]

The thesis that AI can present existential threat likewise has detractors. Skeptics generally state that AGI is not likely in the short-term, or that issues about AGI distract from other problems related to existing AI. [161] Former Google fraud czar Shuman Ghosemajumder considers that for many people beyond the innovation market, existing chatbots and LLMs are already perceived as though they were AGI, leading to further misconception and worry. [162]

Skeptics in some cases charge that the thesis is crypto-religious, with an illogical belief in the possibility of superintelligence changing an illogical belief in an omnipotent God. [163] Some researchers believe that the communication campaigns on AI existential threat by particular AI groups (such as OpenAI, Anthropic, DeepMind, and Conjecture) might be an at effort at regulatory capture and to pump up interest in their products. [164] [165]

In 2023, the CEOs of Google DeepMind, OpenAI and Anthropic, together with other market leaders and scientists, issued a joint statement asserting that "Mitigating the risk of extinction from AI need to be an international concern alongside other societal-scale risks such as pandemics and nuclear war." [152]

Mass unemployment

Researchers from OpenAI approximated that "80% of the U.S. workforce might have at least 10% of their work jobs impacted by the intro of LLMs, while around 19% of workers might see at least 50% of their jobs impacted". [166] [167] They think about workplace employees to be the most exposed, for instance mathematicians, accountants or web designers. [167] AGI might have a much better autonomy, ability to make choices, to user interface with other computer tools, however also to control robotized bodies.

According to Stephen Hawking, the outcome of automation on the quality of life will depend upon how the wealth will be rearranged: [142]

Everyone can take pleasure in a life of glamorous leisure if the machine-produced wealth is shared, or many people can end up miserably poor if the machine-owners effectively lobby versus wealth redistribution. Up until now, the pattern seems to be toward the 2nd alternative, with technology driving ever-increasing inequality

Elon Musk considers that the automation of society will need federal governments to adopt a universal standard income. [168]

See also

Artificial brain - Software and hardware with cognitive capabilities similar to those of the animal or human brain
AI result
AI security - Research area on making AI safe and advantageous
AI positioning - AI conformance to the designated objective
A.I. Rising - 2018 movie directed by Lazar Bodroža
Artificial intelligence
Automated artificial intelligence - Process of automating the application of artificial intelligence
BRAIN Initiative - Collaborative public-private research effort revealed by the Obama administration
China Brain Project
Future of Humanity Institute - Defunct Oxford interdisciplinary research study centre
General video game playing - Ability of artificial intelligence to play different video games
Generative synthetic intelligence - AI system capable of creating material in action to prompts
Human Brain Project - Scientific research study project
Intelligence amplification - Use of information innovation to enhance human intelligence (IA).
Machine ethics - Moral behaviours of man-made makers.
Moravec's paradox.
Multi-task knowing - Solving numerous machine discovering jobs at the exact same time.
Neural scaling law - Statistical law in artificial intelligence.
Outline of artificial intelligence - Overview of and topical guide to synthetic intelligence.
Transhumanism - Philosophical motion.
Synthetic intelligence - Alternate term for or form of synthetic intelligence.
Transfer learning - Artificial intelligence strategy.
Loebner Prize - Annual AI competition.
Hardware for expert system - Hardware specially developed and optimized for synthetic intelligence.
Weak expert system - Form of synthetic intelligence.


Notes

^ a b See below for the origin of the term "strong AI", and see the scholastic definition of "strong AI" and weak AI in the post Chinese space.
^ AI founder John McCarthy writes: "we can not yet characterize in basic what sort of computational treatments we wish to call smart. " [26] (For a conversation of some definitions of intelligence utilized by expert system researchers, see viewpoint of artificial intelligence.).
^ The Lighthill report specifically criticized AI's "grandiose goals" and led the taking apart of AI research study in England. [55] In the U.S., DARPA became figured out to fund just "mission-oriented direct research study, rather than standard undirected research". [56] [57] ^ As AI creator John McCarthy writes "it would be a terrific relief to the remainder of the workers in AI if the inventors of brand-new basic formalisms would reveal their hopes in a more secured kind than has often held true." [61] ^ In "Mind Children" [122] 1015 cps is utilized. More just recently, in 1997, [123] Moravec argued for 108 MIPS which would approximately correspond to 1014 cps. Moravec talks in regards to MIPS, not "cps", which is a non-standard term Kurzweil presented.
^ As defined in a basic AI textbook: "The assertion that devices could possibly act intelligently (or, maybe much better, act as if they were intelligent) is called the 'weak AI' hypothesis by theorists, and the assertion that machines that do so are actually believing (instead of imitating thinking) is called the 'strong AI' hypothesis." [121] ^ Alan Turing made this point in 1950. [36] References

^ Krishna, Sri (9 February 2023). "What is artificial narrow intelligence (ANI)?". VentureBeat. Retrieved 1 March 2024. ANI is created to carry out a single task.
^ "OpenAI Charter". OpenAI. Retrieved 6 April 2023. Our objective is to guarantee that artificial general intelligence benefits all of humankind.
^ Heath, Alex (18 January 2024). "Mark Zuckerberg's brand-new goal is developing artificial general intelligence". The Verge. Retrieved 13 June 2024. Our vision is to develop AI that is much better than human-level at all of the human senses.
^ Baum, Seth D. (2020 ). A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy (PDF) (Report). Global Catastrophic Risk Institute. Retrieved 28 November 2024. 72 AGI R&D tasks were identified as being active in 2020.
^ a b c "AI timelines: What do professionals in artificial intelligence expect for the future?". Our World in Data. Retrieved 6 April 2023.
^ Metz, Cade (15 May 2023). "Some Researchers Say A.I. Is Already Here, Stirring Debate in Tech Circles". The New York Times. Retrieved 18 May 2023.
^ "AI pioneer Geoffrey Hinton quits Google and cautions of danger ahead". The New York Times. 1 May 2023. Retrieved 2 May 2023. It is hard to see how you can avoid the bad actors from utilizing it for bad things.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric (2023 ). "Sparks of Artificial General Intelligence: Early explores GPT-4". arXiv preprint. arXiv:2303.12712. GPT-4 shows stimulates of AGI.
^ Butler, Octavia E. (1993 ). Parable of the Sower. Grand Central Publishing. ISBN 978-0-4466-7550-5. All that you touch you change. All that you alter modifications you.
^ Vinge, Vernor (1992 ). A Fire Upon the Deep. Tor Books. ISBN 978-0-8125-1528-2. The Singularity is coming.
^ Morozov, Evgeny (30 June 2023). "The True Threat of Artificial Intelligence". The New York Times. The genuine danger is not AI itself however the way we deploy it.
^ "Impressed by artificial intelligence? Experts state AGI is coming next, and it has 'existential' risks". ABC News. 23 March 2023. Retrieved 6 April 2023. AGI could present existential risks to humanity.
^ Bostrom, Nick (2014 ). Superintelligence: Paths, Dangers, Strategies. Oxford University Press. ISBN 978-0-1996-7811-2. The very first superintelligence will be the last creation that humankind requires to make.
^ Roose, Kevin (30 May 2023). "A.I. Poses 'Risk of Extinction,' Industry Leaders Warn". The New York City Times. Mitigating the threat of termination from AI ought to be a worldwide top priority.
^ "Statement on AI Risk". Center for AI Safety. Retrieved 1 March 2024. AI specialists warn of danger of termination from AI.
^ Mitchell, Melanie (30 May 2023). "Are AI's Doomsday Scenarios Worth Taking Seriously?". The New York Times. We are far from creating devices that can outthink us in basic methods.
^ LeCun, Yann (June 2023). "AGI does not provide an existential risk". Medium. There is no reason to fear AI as an existential risk.
^ Kurzweil 2005, p. 260.
^ a b Kurzweil, Ray (5 August 2005), "Long Live AI", Forbes, archived from the original on 14 August 2005: Kurzweil describes strong AI as "device intelligence with the complete variety of human intelligence.".
^ "The Age of Artificial Intelligence: George John at TEDxLondonBusinessSchool 2013". Archived from the original on 26 February 2014. Retrieved 22 February 2014.
^ Newell & Simon 1976, This is the term they use for "human-level" intelligence in the physical symbol system hypothesis.
^ "The Open University on Strong and Weak AI". Archived from the original on 25 September 2009. Retrieved 8 October 2007.
^ "What is synthetic superintelligence (ASI)?|Definition from TechTarget". Enterprise AI. Retrieved 8 October 2023.
^ "Artificial intelligence is transforming our world - it is on everyone to ensure that it goes well". Our World in Data. Retrieved 8 October 2023.
^ Dickson, Ben (16 November 2023). "Here is how far we are to achieving AGI, according to DeepMind". VentureBeat.
^ McCarthy, John (2007a). "Basic Questions". Stanford University. Archived from the original on 26 October 2007. Retrieved 6 December 2007.
^ This list of smart traits is based upon the subjects covered by major AI textbooks, consisting of: Russell & Norvig 2003, Luger & Stubblefield 2004, Poole, Mackworth & Goebel 1998 and Nilsson 1998.
^ Johnson 1987.
^ de Charms, R. (1968 ). Personal causation. New York City: Academic Press.
^ a b Pfeifer, R. and Bongard J. C., How the body shapes the way we think: a brand-new view of intelligence (The MIT Press, 2007). ISBN 0-2621-6239-3.
^ White, R. W. (1959 ). "Motivation reevaluated: The concept of competence". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ White, R. W. (1959 ). "Motivation reassessed: The concept of skills". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ Muehlhauser, Luke (11 August 2013). "What is AGI?". Machine Intelligence Research Institute. Archived from the initial on 25 April 2014. Retrieved 1 May 2014.
^ "What is Artificial General Intelligence (AGI)?|4 Tests For Ensuring Artificial General Intelligence". Talky Blog. 13 July 2019. Archived from the initial on 17 July 2019. Retrieved 17 July 2019.
^ Kirk-Giannini, Cameron Domenico; Goldstein, Simon (16 October 2023). "AI is closer than ever to passing the Turing test for 'intelligence'. What happens when it does?". The Conversation. Retrieved 22 September 2024.
^ a b Turing 1950.
^ Turing, Alan (1952 ). B. Jack Copeland (ed.). Can Automatic Calculating Machines Be Said To Think?. Oxford: Oxford University Press. pp. 487-506. ISBN 978-0-1982-5079-1.
^ "Eugene Goostman is a genuine young boy - the Turing Test says so". The Guardian. 9 June 2014. ISSN 0261-3077. Retrieved 3 March 2024.
^ "Scientists challenge whether computer 'Eugene Goostman' passed Turing test". BBC News. 9 June 2014. Retrieved 3 March 2024.
^ Jones, Cameron R.; Bergen, Benjamin K. (9 May 2024). "People can not identify GPT-4 from a human in a Turing test". arXiv:2405.08007 [cs.HC]
^ Varanasi, Lakshmi (21 March 2023). "AI models like ChatGPT and GPT-4 are acing whatever from the bar test to AP Biology. Here's a list of difficult tests both AI variations have passed". Business Insider. Retrieved 30 May 2023.
^ Naysmith, Caleb (7 February 2023). "6 Jobs Expert System Is Already Replacing and How Investors Can Take Advantage Of It". Retrieved 30 May 2023.
^ Turk, Victoria (28 January 2015). "The Plan to Replace the Turing Test with a 'Turing Olympics'". Vice. Retrieved 3 March 2024.
^ Gopani, Avi (25 May 2022). "Turing Test is undependable. The Winograd Schema is outdated. Coffee is the response". Analytics India Magazine. Retrieved 3 March 2024.
^ Bhaimiya, Sawdah (20 June 2023). "DeepMind's co-founder recommended evaluating an AI chatbot's ability to turn $100,000 into $1 million to measure human-like intelligence". Business Insider. Retrieved 3 March 2024.
^ Suleyman, Mustafa (14 July 2023). "Mustafa Suleyman: My new Turing test would see if AI can make $1 million". MIT Technology Review. Retrieved 3 March 2024.
^ Shapiro, Stuart C. (1992 ). "Expert System" (PDF). In Stuart C. Shapiro (ed.). Encyclopedia of Artificial Intelligence (Second ed.). New York City: John Wiley. pp. 54-57. Archived (PDF) from the original on 1 February 2016. (Section 4 is on "AI-Complete Tasks".).
^ Yampolskiy, Roman V. (2012 ). Xin-She Yang (ed.). "Turing Test as a Specifying Feature of AI-Completeness" (PDF). Expert System, Evolutionary Computation and Metaheuristics (AIECM): 3-17. Archived (PDF) from the original on 22 May 2013.
^ "AI Index: State of AI in 13 Charts". Stanford University Human-Centered Expert System. 15 April 2024. Retrieved 27 May 2024.
^ Crevier 1993, pp. 48-50.
^ Kaplan, Andreas (2022 ). "Artificial Intelligence, Business and Civilization - Our Fate Made in Machines". Archived from the initial on 6 May 2022. Retrieved 12 March 2022.
^ Simon 1965, p. 96 quoted in Crevier 1993, p. 109.
^ "Scientist on the Set: An Interview with Marvin Minsky". Archived from the original on 16 July 2012. Retrieved 5 April 2008.
^ Marvin Minsky to Darrach (1970 ), quoted in Crevier (1993, p. 109).
^ Lighthill 1973; Howe 1994.
^ a b NRC 1999, "Shift to Applied Research Increases Investment".
^ Crevier 1993, pp. 115-117; Russell & Norvig 2003, pp. 21-22.
^ Crevier 1993, p. 211, Russell & Norvig 2003, p. 24 and see likewise Feigenbaum & McCorduck 1983.
^ Crevier 1993, pp. 161-162, 197-203, 240; Russell & Norvig 2003, p. 25.
^ Crevier 1993, pp. 209-212.
^ McCarthy, John (2000 ). "Reply to Lighthill". Stanford University. Archived from the original on 30 September 2008. Retrieved 29 September 2007.
^ Markoff, John (14 October 2005). "Behind Expert system, a Squadron of Bright Real People". The New York Times. Archived from the initial on 2 February 2023. Retrieved 18 February 2017. At its low point, some computer system researchers and software engineers avoided the term expert system for fear of being deemed wild-eyed dreamers.
^ Russell & Norvig 2003, pp. 25-26
^ "Trends in the Emerging Tech Hype Cycle". Gartner Reports. Archived from the original on 22 May 2019. Retrieved 7 May 2019.
^ a b Moravec 1988, p. 20
^ Harnad, S. (1990 ). "The Symbol Grounding Problem". Physica D. 42 (1-3): 335-346. arXiv: cs/9906002. Bibcode:1990 PhyD ... 42..335 H. doi:10.1016/ 0167-2789( 90 )90087-6. S2CID 3204300.
^ Gubrud 1997
^ Hutter, Marcus (2005 ). Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Texts in Theoretical Computer Science an EATCS Series. Springer. doi:10.1007/ b138233. ISBN 978-3-5402-6877-2. S2CID 33352850. Archived from the initial on 19 July 2022. Retrieved 19 July 2022.
^ Legg, Shane (2008 ). Machine Super Intelligence (PDF) (Thesis). University of Lugano. Archived (PDF) from the initial on 15 June 2022. Retrieved 19 July 2022.
^ Goertzel, Ben (2014 ). Artificial General Intelligence. Lecture Notes in Computer Technology. Vol. 8598. Journal of Artificial General Intelligence. doi:10.1007/ 978-3-319-09274-4. ISBN 978-3-3190-9273-7. S2CID 8387410.
^ "Who coined the term "AGI"?". goertzel.org. Archived from the initial on 28 December 2018. Retrieved 28 December 2018., through Life 3.0: 'The term "AGI" was promoted by ... Shane Legg, Mark Gubrud and Ben Goertzel'
^ Wang & Goertzel 2007
^ "First International Summer School in Artificial General Intelligence, Main summer season school: June 22 - July 3, 2009, OpenCog Lab: July 6-9, 2009". Archived from the original on 28 September 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2009/2010 - пролетен триместър" [Elective courses 2009/2010 - spring trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the original on 26 July 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2010/2011 - зимен триместър" [Elective courses 2010/2011 - winter season trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the initial on 26 July 2020. Retrieved 11 May 2020.
^ Shevlin, Henry; Vold, Karina; Crosby, Matthew; Halina, Marta (4 October 2019). "The limitations of maker intelligence: Despite development in machine intelligence, artificial general intelligence is still a major challenge". EMBO Reports. 20 (10 ): e49177. doi:10.15252/ embr.201949177. ISSN 1469-221X. PMC 6776890. PMID 31531926.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric; Kamar, Ece; Lee, Peter; Lee, Yin Tat; Li, Yuanzhi; Lundberg, Scott; Nori, Harsha; Palangi, Hamid; Ribeiro, Marco Tulio; Zhang, Yi (27 March 2023). "Sparks of Artificial General Intelligence: Early explores GPT-4". arXiv:2303.12712 [cs.CL]
^ "Microsoft Researchers Claim GPT-4 Is Showing "Sparks" of AGI". Futurism. 23 March 2023. Retrieved 13 December 2023.
^ Allen, Paul; Greaves, Mark (12 October 2011). "The Singularity Isn't Near". MIT Technology Review. Retrieved 17 September 2014.
^ Winfield, Alan. "Artificial intelligence will not become a Frankenstein's monster". The Guardian. Archived from the initial on 17 September 2014. Retrieved 17 September 2014.
^ Deane, George (2022 ). "Machines That Feel and Think: The Role of Affective Feelings and Mental Action in (Artificial) General Intelligence". Artificial Life. 28 (3 ): 289-309. doi:10.1162/ artl_a_00368. ISSN 1064-5462. PMID 35881678. S2CID 251069071.
^ a b c Clocksin 2003.
^ Fjelland, Ragnar (17 June 2020). "Why basic expert system will not be realized". Humanities and Social Sciences Communications. 7 (1 ): 1-9. doi:10.1057/ s41599-020-0494-4. hdl:11250/ 2726984. ISSN 2662-9992. S2CID 219710554.
^ McCarthy 2007b.
^ Khatchadourian, Raffi (23 November 2015). "The Doomsday Invention: Will artificial intelligence bring us utopia or destruction?". The New Yorker. Archived from the original on 28 January 2016. Retrieved 7 February 2016.
^ Müller, V. C., & Bostrom, N. (2016 ). Future progress in expert system: A survey of professional opinion. In Fundamental concerns of expert system (pp. 555-572). Springer, Cham.
^ Armstrong, Stuart, and Kaj Sotala. 2012. "How We're Predicting AI-or Failing To." In Beyond AI: Artificial Dreams, edited by Jan Romportl, Pavel Ircing, Eva Žáčková, Michal Polák and Radek Schuster, 52-75. Plzeň: University of West Bohemia
^ "Microsoft Now Claims GPT-4 Shows 'Sparks' of General Intelligence". 24 March 2023.
^ Shimek, Cary (6 July 2023). "AI Outperforms Humans in Creativity Test". Neuroscience News. Retrieved 20 October 2023.
^ Guzik, Erik E.; Byrge, Christian; Gilde, Christian (1 December 2023). "The originality of machines: AI takes the Torrance Test". Journal of Creativity. 33 (3 ): 100065. doi:10.1016/ j.yjoc.2023.100065. ISSN 2713-3745. S2CID 261087185.
^ Arcas, Blaise Agüera y (10 October 2023). "Artificial General Intelligence Is Already Here". Noema.
^ Zia, Tehseen (8 January 2024). "Unveiling of Large Multimodal Models: Shaping the Landscape of Language Models in 2024". Unite.ai. Retrieved 26 May 2024.
^ "Introducing OpenAI o1-preview". OpenAI. 12 September 2024.
^ Knight, Will. "OpenAI Announces a Brand-new AI Model, Code-Named Strawberry, That Solves Difficult Problems Step by Step". Wired. ISSN 1059-1028. Retrieved 17 September 2024.
^ "OpenAI Employee Claims AGI Has Been Achieved". Orbital Today. 13 December 2024. Retrieved 27 December 2024.
^ "AI Index: State of AI in 13 Charts". hai.stanford.edu. 15 April 2024. Retrieved 7 June 2024.
^ "Next-Gen AI: OpenAI and Meta's Leap Towards Reasoning Machines". Unite.ai. 19 April 2024. Retrieved 7 June 2024.
^ James, Alex P. (2022 ). "The Why, What, and How of Artificial General Intelligence Chip Development". IEEE Transactions on Cognitive and Developmental Systems. 14 (2 ): 333-347. arXiv:2012.06338. doi:10.1109/ TCDS.2021.3069871. ISSN 2379-8920. S2CID 228376556. Archived from the original on 28 August 2022. Retrieved 28 August 2022.
^ Pei, Jing; Deng, Lei; Song, S