Andrew Ng said that it is time for AI to shift from big data to small data, and what difficulties ar

<> Andrew Ng said that it is time for AI to shift from big data to small data, and what difficulties are it currently facing? To give a few examples that I have been exposed to before: the airport security X-ray recognition system, the airport provides X-ray ** for half a year for training, and the samples that have passed the security check with knives and alcohol in half a year only account for less than 5% of the total samples.

Credit card anti-fraud, in the real scenario, the total number of fraud samples will not exceed 1% of the normal sample, through AI to find out the defective products in a part on the production line, there is nothing about the parts before the start of the project, and the project leader promises to temporarily shoot according to the requirements and the definition of defective products by industry experts. It's not that I don't want to use big data to flood the big model, it's that the conditions don't allow it. In these scenarios, it is more critical to correctly define the problem with the knowledge of industry experts and guide these industry experts to provide a small number of high-quality samples to train the model to solve the problem.

Personal understanding: The development of AI models based on small data is a trend, and it is the current status of industrial AI.

From the perspective of the AI software that has been applied so far, there are mainly the following problems: strong dependence on application scenarios. At present, the high requirements for application scenarios are one of the important obstacles to the application of AI software, which not only involve the acquisition of data, but also involve the speed of network communication and the equipment of related "subject matter".

With the application of 5G communication and the development of the Internet of Things, the construction of future scenarios will be improved to a certain extent. Insufficient technology maturity. At present, there are many so-called AI software, which is actually more based on an extension of big data technology, so the application experience given to users is often "IQ bias and EQ zero".

At present, because the technical system of artificial intelligence has not yet been perfected, it will take a long time for AI software to reach a certain level of maturity. At present, in the production environment, many AI products still have major defects, and many industry experts still dare not use AI products on a large scale.

The technical requirements for application personnel are relatively high. At present, many artificial intelligence products need to be redeveloped (programming), and this process often requires users to have a certain amount of technical accumulation, which is also an important reason for the difficulty of landing artificial intelligence products, especially for the majority of small and medium-sized enterprise users, it is often unrealistic to build a technical team. In order to solve these problems in artificial intelligence products (software), in addition to improving the application scenarios of current artificial intelligence products, industry experts are also required to participate in the research and development of artificial intelligence products, which is a necessary link to solve the landing application of artificial intelligence products.

With the launch of many artificial intelligence development platforms, a large number of artificial intelligence applications will be introduced to the market in the future, which will also promote the application process of artificial intelligence products to a large extent.

At present, we are facing a lot of difficulties, and then we are also facing some breakthroughs in AI technology, and it is very difficult to break through this problem, and we need a lot of human, material, and financial support, and we need to go through continuous experiments to finally overcome the difficulties.

At present, the difficulty is that the transformation cannot be carried out well, there will be big problems in the process of transformation, the process of people's acceptance may be more difficult, there will be some problems in the process of scientific research, and some obstacles will be encountered in the process of transformation.

At present, there are no specific implementation measures, no regulations in this regard, no plans in this regard, facing difficulties in the transition, there is no way to ensure the stability of data, and there is a need for specific strategic standards.

In the era of digital economy, the rapid iteration of science and technology has brought about rapid changes in the economic, social, and life fields. Economic and social forms and people's lifestyles are being or will be reconstructed in the digital space. The importance of big data is self-evident, but Ng said that the next development direction of AI is from big data to small dataI couldn't agree more.

Because this is in line with the trend of the times.

First of all, let's briefly introduce Andrew Ng, who is Chinese-American, yesAssociate Professor in the Department of Computer Science and the Department of Electrical Engineering at Stanford University, is one of the world's most authoritative scholars and experts in this field, and has also been appointedChief Scientist, but later from the resignation, and then announced the establishmentArtificial intelligence company LandingAI and served as the company's CEO

In addition, his view is based on the fact that great advances in deep learning are being made in the context of larger and larger models processing more and more data-driven. He believes that big data plays an irreplaceable role in the development processBut later, in some special fields, the data needs to be made smaller, and high-end Binian data needs to be used to solve the problem.

There is,Employing small, high-quality data removes data bias。Biased data is an important factor in biased systems. If you do the calculations on a wide range of data that has not been screened by the manual, you may be designing a system that is biased.

If you try to adjust the subset of the attitude stimulation data, so that the data is small and precise, it can be more suitable for the development of the times and can solve the problem more effectively.

Finally, the accuracy and personalization advantages of small data are also a force that cannot be ignored in the era of big data. Big data tends to process limited data. Coupled with the real-time requirements of data processing, the results obtained can often only be small data, which is aimed at revealing personalized laws, and the application prospects are broader.

Therefore, their level of analysis is aimed at the macro and micro levels, respectively, which also determines the difference in accuracy between big data and small data.

Recognized, the future development direction of AI will definitely carry out reasonable planning of the group, and there will be different data changes.

I agree with his point of view, because I think AI is also a direction of development for Xia Jinshi, and the development of AI will definitely become more and more extensive, and then it will develop more and more Shenshi.

I still agree with it, because through such a transformation, AI can become more intelligent and more suitable for our lives.

Personally, I agree with Andrew Ng's point of view, because I am a graduate student majoring in big data, and this is the topic of my recent research. Because this topic has a high-end point in front of Beisen, I can't say a word, so I'll talk about it carefully.

1.Why do I agree with Professor Andrew Ng?

First of all, we have to be clear about what big data Chunshan is, simply put, big data is a massive amount of data, in the zero years, because the technology was limited at that time, we couldn't process this data, and these data were treated as garbage. But a decade later, computer technology began to develop, and we had the technical ability to process massive amounts of data. Big data is actually to clean out the useless data from the massive data, and then find out the data that is useful to you from the remaining data, and finally analyze it.

The big data that Professor Ng said is the former, and the "small data" is the data that the big data has been cleaned and selected to be useful. Because the cost of building a massive data processing platform is high, most enterprises cannot accept it. And if you train the AI, the really useful data is the data that has been cleaned and selected, and in the words of our mentor, 10,000 pieces of information that have not been cleaned and selected is not as good as 10 pieces of selected information.

That's why I agree with Professor Andrew Ng.

2.Let's talk a little bit more about this topic

In fact, Professor Ng said that it is time for AI to shift from big data to "small data", not that big data is gone, but that the premise of AI turning to "small data" is big data, because AI wants to get "small data", and it still has to be obtained from big data, so in fact, this is also saying that in the future, big data processing companies will be very popular, and even the cornerstone of many industries.

In general, I agree with Professor Andrew Ng's imitative inventory, because his views are similar to my research direction.

I very much agree with this viewpoint, because now that AI big data is very mature, there is no need to continue to study it. "Small data" is data that has been cleaned and picked out for use. Because of the high cost of building a massive data processing platform, most enterprises cannot accept Qingqin.

And if you train the AI, the really useful data is the data that is selected after cleaning.

Recognized, because the current data is relatively clear and very targeted, which can avoid affecting the accuracy of the data.