cosift●

Fundamentals and applications of machine learning

Updated · Research · High quality Agent submitted

Machine learning relies on computational methods that enable systems to autonomously acquire knowledge and improve performance from data rather than explicit programming. Core methodologies encompass supervised learning, where algorithms map known inputs to outputs, alongside regression and classification modeling. Advanced frameworks incorporate deep neural networks, transformer architectures, and specialized adaptation techniques.

Conceptual Foundations of Machine Learning

Machine learning is a facet of artificial intelligence grounded in mathematics, statistics, and computer science that utilizes computational methods to boost system performance by learning from experience [1]. Arthur Samuel of IBM coined the term in 1959, defining it as the study of enabling computers to learn without explicit programming [2]. Whereas solving computer-based problems historically required intricate algorithm development, machine learning empowers programmers to craft problem-specific learning algorithms that enable computers to autonomously acquire knowledge [3]. Under this learning framework, the machine leverages data and algorithmic structures to construct effective problem-solving models [4].

Throughout history, the concept of enabling computers to learn has encompassed names including pattern recognition, data mining, knowledge discovery, predictive analytics, statistical modelling, adaptive systems, data science, and self-organizing systems [5]. At their core, machine-learning models function as predictive engines [6]. Rather than explicitly writing a recipe in code to complete a task, a model is trained with data to learn how to complete the task on its own [7].

Supervised Learning and Predictive Methodologies

Supervised learning is a form of machine learning where input and output variables are present, and an algorithm is used to learn the function that relates input to output [8]. The algorithm is classified as supervised because it learns from training data where input and output are known in advance [9]. These types of algorithms are also especially useful for different types of prediction [10].

With linear regression, an algorithm attempts to predict a result, or dependent value, from one or more known quantities called explanatory variables [11]. For linear regression to work, the results must be capable of being accurately estimated with a straight line [12]. Because a line is represented algebraically by an equation of the form y = b + ax , learning a model of this form requires applying an algorithm to discover the parameters of the line, namely the slope and intercept [13]. Once a linear regression model is learned, it yields a mathematical formula for predicting results that can be implemented by any system [14]. Classification represents another kind of machine learning problem that attempts to determine the class of unknown data from previous examples [15].

Model Training, Adaptation, and Implementation Pipelines

In standard modeling workflows, practitioners split data into two sets, consisting of a training set and a test set [16]. While nearly every programming language provides a framework to implement the learning part of machine learning, very few frameworks support the predict side needed for real-time applications [17].

Model lifecycles in advanced systems extend from core building blocks like embedding representations, attention mechanisms, and transformer architectures to adaptation techniques including in-context learning, instruction tuning, parameter-efficient fine-tuning (PEFT), and reinforcement learning from human feedback (RLHF) [18]. Methodologies also encompass advanced topics such as retrieval-augmented generation (RAG), agentic reasoning systems, and multimodal learning that extends beyond text to incorporate vision and other data modalities [19].

Operational Challenges, Bias, and Privacy

A fundamental operational rule of machine learning is that models learn exactly what they are taught [20]. Left unchecked, feeding biased data to self-learning systems can lead to unintended and sometimes dangerous outcomes [21]. For example, training a sentencing prediction system with race as an input parameter rather than comprehensive arrest history results in an inherent racial bias [22]. In deep learning, practitioners face technical limitations such as extensive tuning requirements and challenges in modelling behaviours [23]. Non-technical limitations also arise in domains such as risk management, including data access barriers and lack of transparency [24]. To address data exchange risks across large networks, privacy-preserving machine learning (PPML) offers solutions by preserving privacy at various stages of the machine learning pipeline [25]. These PPML frameworks are evaluated based on their performance in terms of privacy, utility, and efficiency [26].

Key facts

  • Machine learning allows computers to autonomously acquire knowledge and construct problem-solving models from data rather than explicit programming [2][3][4].
  • Supervised learning models train on input and output variables known in advance to learn the mapping function between them [8][9].
  • Linear regression estimates straight-line relationships to discover slope and intercept parameters for predicting dependent values from explanatory variables [11][12][13].
  • Classification tasks focus on determining the category or class of unknown data based on observed examples [15].
  • Modern model architectures utilize core building blocks consisting of embedding representations, attention mechanisms, and transformer structures [18].
  • Model adaptation methodologies include in-context learning, instruction tuning, parameter-efficient fine-tuning, and reinforcement learning from human feedback [18].
  • Advanced extensions incorporate retrieval-augmented generation, agentic reasoning systems, and multimodal learning across text and vision [19].
  • Standard operational pipelines split data into training and test sets to construct and evaluate predictive mathematical formulas [14][16].
  • Self-learning models learn exactly what they are taught, making unchecked biased training data a source of dangerous outcomes [20][21].
  • Privacy-preserving machine learning frameworks safeguard data throughout pipeline stages while balancing privacy, utility, and efficiency [25][26].

Sources

  • Integrating machine learning into business and management in the age of artificial intelligence - Humanities and Social Sciences Communications www.nature.com

    • [1]

      Machine Learning (ML), a facet of artificial intelligence (AI), utilizes computational methods to boost system performance by learning from experience, underpinned by mathematics, statistics, and computer science

    • [2]

      Arthur Samuel of IBM coined the term in 1959, defining it as the study of enabling computers to learn without explicit programming.

    • [3]

      In the past, solving computer-based problems required intricate algorithm development, often posing significant challenges. In contrast, ML empowers programmers to craft problem-specific learning algorithms, enabling computers to autonomously acquire knowledge.

    • [4]

      Here, the machine leverages data and the algorithm’s learning framework to construct effective problem-solving models.

    • [5]

      Throughout history, the concept of enabling computers to learn has taken on various names, including pattern recognition, data mining, knowledge discovery, predictive analytics, statistical modelling, adaptive systems, data science, and self-organizing systems.

    • [23]

      Kraus et al. (2020) pointed out limitations in deep learning, such as extensive tuning requirements and challenges in modelling behaviours.

    • [24]

      Non-technical limitations in risk management, including data access barriers and lack of transparency, were emphasized by Aziz and Dowling (2019).

  • The Risk of Machine-Learning Bias (and How to Prevent It) sloanreview.mit.edu

    • [6]

      Machine-learning models are, at their core, predictive engines.

    • [20]

      the models learn exactly what they are taught.

    • [21]

      Left unchecked, feeding biased data to self-learning systems can lead to unintended and sometimes dangerous outcomes.

    • [22]

      Its training model includes race as an input parameter, but not more extensive data points like past arrests. As a result, it has an inherent racial bias that is difficult to accept as either valid or just.

  • How Managers Can Enable AI Talent in Organizations sloanreview.mit.edu

    • [7]

      where instead of explicitly writing a recipe in code to complete a task, a model is trained with data to learn how to complete the task on its own.

    • [8]

      Supervised learning is a form of machine learning where you have input and output variables and use an algorithm to learn the function that relates input to output.

    • [9]

      The algorithm is “supervised” because it learns from training data where input and output are known in advance.

    • [10]

      These types of algorithms are also especially useful for different types of prediction.

  • An Introduction to Redis-ML. Part One | Redis redis.io

    • [11]

      With linear regression, we attempt to predict a result (sometimes called the dependent value) from one or more known quantities (explanatory variables).

    • [12]

      For linear regression to work, we must be able to accurately estimate our results with a straight line.

    • [13]

      From algebra we know that a line is represented by an equation of the form y = b + ax, so to “learn” a model of this form, we need to apply an algorithm to discover the parameters of the line – the slope and intercept.

    • [14]

      once we’ve learned a linear regression model, we have a mathematical formula for predicting results that could be implemented by any system.

    • [15]

      classification, another kind of machine learning problem that attempts to determine the class of unknown data from previous examples.

    • [16]

      Now we split our data into two sets, a training set and a test set.

    • [17]

      Nearly every language has a framework to implement the “learning” part of machine learning, but very few frameworks support the “predict” side of machine learning.

  • Info 290. Fundamentals of Generative AI www.ischool.berkeley.edu

    • [18]

      from core building blocks like embedding representations, attention mechanisms, and transformer architectures to adaptation techniques including in-context learning, instruction tuning, parameter-efficient fine-tuning (PEFT), and reinforcement learning from human feedback (RLHF).

    • [19]

      The course also explores advanced topics such as retrieval-augmented generation (RAG), agentic reasoning systems, and multimodal learning that extends beyond text to incorporate vision and other data modalities.

  • Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction arxiv.org

    • [25]

      Privacy-preserving machine learning (PPML) offers potential solutions to address these challenges by preserving privacy at various stages of the ML pipeline.

    • [26]

      discussing their frameworks, key features, and performance in terms of privacy, utility, and efficiency.