Artificial intelligence (AI) algorithms are typically used to solve the most challenging problems, achieving state-of-the-art performances and providing a substantial impact on all aspects of human life. However, because of their computational complexity and high number of parallel processing operations, AI algorithms require significant computational resources (processing power and memory), they consume a considerable amount of energy and require powerful hardware (such as GPUs, servers, clouds). All of these constraints, significantly limit their applicability. In recent years, there has been an increasing need for implementation of AI algorithms on devices with limited resources (memory space, processing power and available energy). Therefore, the problem of implementing complex AI algorithms on devices with limited resources is one of the most topical and challenging research problems in the field of AI. This project deals with solving the mentioned problem, where the overall objective is to propose and implement new approaches to reduce the complexity of AI algorithms using advanced quantization and compression methods, based on the strong expertise of the project team in the field. An important research question we will answer is how to make state-of-the-art DNN, with numerous parameters, more compact and efficient. We will exploit the fact that reduced precision of the DNN parameters, i.e. going from 32-bit floating point to a low-bit fixed point, can be achieved by thoughtful application of quantization and compression, which will result in a worthy compression ratio and, accordingly in the reduced storage and energy cost, and computational requirements. Majority of the research directions has been set toward achieving the highest possible compression ratio of DNN parameters without significant accuracy loss. Since this field of research is still in an early stage, significant improvements are possible. This project offers new approaches to advanced quantization and compression of DNN parameters in order to outperform state-of-the-art solutions. Besides their fundamental scientific importance in the field of AI, all theoretical results will also be fully applicable in practice. Hence, an important part of the project concept is to validate all the proposed theoretical results and to examine their applicability. This will be done by implementing all theoretical solutions on the developed DNN, as a part of the target use case. Eventually, all theoretical results will be implemented and validated and their performance will be evaluated within a real industrial application using data from a real industrial environment.
Very topical issues present in generally very powerful AI algorithms related to decreasing computational complexity and memory resources that are of particular importance in portable and edge computing devices with limited memory and processing power are the driving force behind this project proposal. Specifically, this project contributes to a new and fastly growing research in the worldwide AI science solving these particular issues by prudent and deliberate application of quantization theory. Due to the topicality of these issues and the fact that related research is still in the early stage and deserves further investigation, our project team will explore and propose innovative methods of compression and quantization of DNN parameters (weights, biases, activations) and deep features. Also, the goal of the project is to develop a state-of-the-art deep neural network model with a high performance not only on the hardware usually used for AI applications but also on devices with limited computational resources and thus enabling them to support energy demanding and memory constraint applications. In order to achieve these goals, the project proposes an integrated approach to quantization and compression of DNN parameters, based on statistical modeling of the data per layers, as well as of the data subsets within layers and adaptation of the quantizers itself on the statistical characteristics of the input data. Moreover, from the exploration of the compression and quantization effects in DNNs vis-à-vis its accuracy, the benchmark of our methodology will be defined. The researchers’ interdisciplinary competence ensures successful development of innovative quantization, compression and learning methodologies that will enable reducing the complexity of AI algorithms and its much wider usage. The results obtained within this project will find a wide range of applications in both academia and industry, particularly in numerous latency-critical services.
The project team is fully compatible, since all team members have basic knowledge of quantization and coding theory, as well as of neural networks and AI algorithms, which will greatly facilitate their collaboration and communication and will improve work efficiency.
Moreover, the project team has a great complementarity since each team member has a unique deep knowledge and strong expertise required for the project realization.
Team members have been working successfully at the University of Niš, Faculty of Electronic Engineering, on joint scientific projects and papers for years and therefore know each other’s strengths and possibilities very well, which will give the optimal outputs of this project.
Professor Zoran Perić as a PI has a stunning academic and scientific career, as well as necessary management and administrative skills to lead this project. With years of experience and significant scientific and applicable results, a Full Professor Dejan Ćirić will give great contribution in the activities within this project related to AI (DNN design, application and evaluation), audio signal processing and sound acquisition. The special expertise of Associate Professor Aleksandra Jovanović is design of vector quantizers and also design of power efficient signal constellations. Therefore, it is expected that she will predominantly contribute to the part of the project related to data clustering and classification where the multidimensional space partition is an important issue. Assistant Professor Milan Dinčić has strong expertise in the design of VLC (variable-length coding) compression algorithms as well as in the statistical modeling of input data for DNNs and other AI algorithms. His contribution mostly will be in the domain of compression. Associate Professor Jelena Nikolić will contribute to this project with her expertise in speech compression and its application in machine learning and AI. In particular, she will give great contribution in the project activities related to quantization and compression of DNN parameters, where her expertise can be exploited greatly. Ph.D. student Nikola Vučić as a young researcher brings fresh energy to the project and provides support to the part related to quantization and compression. Ph.D. student Bojan Denić will contribute to the part of the project related to data classification, as it is closely related to his research areas. Professor Vladimir Despotović is an expert in machine learning, natural language processing and fractional calculus. His knowledge and extensive international experience considerably increase the capacity of the team to develop advanced methods of learning in AI. All project objectives will be realized through an interdisciplinary approach and knowledge synergy of all team members. Eventually, team members are carefully selected so that everyone has a precisely defined role in the project.
Members of our project team have participated in several projects closely related to the topic of this project. PI, P2 and P4 have been involved in the project Human-Machine Speech Communication, related to a detailed analysis of all problems in speech to machine communication in both directions with the aim to give machines intelligence and human-like capability to speak and understand speech. Also, PI, P2 and P4 have been involved in the project The Development of Dialogue Systems for Serbian and Other South Slavic Languages considered the development of dialogue systems using artificial neural networks, which is highly correlated with this project. PI, P5 and P6 have taken part in the bilateral project, Fractional calculus approach to machine learning, between Serbia and Slovakia. The project developed a novel approach to machine learning by introducing fractional-order calculus to optimization methods used in machine learning algorithms. P1 was a project leader of recently finished project SONO360 - Smart audio interface financed by the Innovation Fund of the Republic of Serbia. The project dealt with the development of a prototype of smart audio device capable of acquiring sound from 3D space applying direction of arrival (DoA) and beamforming technique using spherical microphone array, as well as detecting and recognizing domestic sound events based on a specific design of a DNN. There is a close relation to this project with regards to DNN design and implementation in the field of sound. The experience and knowledge gained working on those projects are very valuable for the accomplishment of the tasks and goals we have defined in this project.
October 9th 2020
The project Com-in-AI was presented on October 9th 2020 at Kalemegdan (Belgrade) within the Science Fund exposition dedicated to the promotion of the Program for the development of projects in the domain of AI. More details at link.
The dissemination, exploitation and communication activities will be carried out continuously throughout the lifetime of the project, but also after the project completion, to ensure long-term effects of the project. The dissemination activities will be focused on promoting the scientific results, the potential usage and also the project in general among a wide scientific community as well as among the target groups relevant for exploitation of the achieved results. Workshops to be organized will target the scientific and the interested non-scientific stakeholders (industry, AI companies, Public authorities at national and local levels, relevant decision makers) to exchange ideas on the scientific, economic and social aspects as well as applicability of the project results and to discuss possible collaboration.
Here you can find a list of sorted literature relevant to the project topic:
>>> Link to the list of relevant literature <<<
This research was supported by the Science Fund of the Republic of Serbia, 6527104, AI- Com-in-AI.
This website is made with the financial support of the Science Fund of the Republic of Serbia.
For the website's content is only responsible University of Niš, Faculty of Electronic Engineering and the content doesn't represent the attitudes of the Science Fund of the Republic of Serbia.