MATLAB

From Wikipedia, the free encyclopedia

MATLAB

L-shaped membrane logo
MATLAB R2013a running on Windows 8
Developer(s)	MathWorks
Initial release	1984; 34 years ago
Stable release	R2018b / September 12, 2018; 2 months ago
Preview release	None [±]
Written in	C, C++, Java
Operating system	Windows, macOS, and Linux
Platform	IA-32, x86-64
Type	Numerical computing
License	Proprietary commercial software
Website	mathworks.com

MATLAB

Paradigm	multi-paradigm: functional, imperative, procedural, object-oriented, array
Designed by	Cleve Moler
Developer	MathWorks
First appeared	late 1970s
Stable release	9.4 (R2018a) / March 14, 2018; 8 months ago
Preview release	None [±]
Typing discipline	dynamic, weak
Filename extensions	.m
Website	mathworks.com
Influenced by
APL EISPACK LINPACK PL/0 Speakeasy
Influenced
Julia Octave Scilab
MATLAB Programming at Wikibooks

MATLAB (matrix laboratory) is a multi-paradigm numerical computing environment and proprietary programming language developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, C#, Java, Fortran and Python.

Although MATLAB is intended primarily for numerical computing, an optional toolbox uses the MuPAD symbolic engine, allowing access to symbolic computing abilities. An additional package, Simulink, adds graphical multi-domain simulation and model-based design for dynamic and embedded systems.

As of 2018, MATLAB has more than 3 million users worldwide. MATLAB users come from various backgrounds of engineering, science, and economics.

History

Cleve Moler, the chairman of the computer science department at the University of New Mexico, started developing MATLAB in the late 1970s. He designed it to give his students access to LINPACK and EISPACK without them having to learn Fortran. It soon spread to other universities and found a strong audience within the applied mathematics community. Jack Little, an engineer, was exposed to it during a visit Moler made to Stanford University in 1983. Recognizing its commercial potential, he joined with Moler and Steve Bangert. They rewrote MATLAB in C and founded MathWorks in 1984 to continue its development. These rewritten libraries were known as JACKPAC. In 2000, MATLAB was rewritten to use a newer set of libraries for matrix manipulation, LAPACK.

MATLAB was first adopted by researchers and practitioners in control engineering, Little's specialty, but quickly spread to many other domains. It is now also used in education, in particular the teaching of linear algebra, numerical analysis, and is popular amongst scientists involved in image processing.

Syntax

The MATLAB application is built around the MATLAB scripting language. Common usage of the MATLAB application involves using the Command Window as an interactive mathematical shell or executing text files containing MATLAB code.

Variables

Variables are defined using the assignment operator, =. MATLAB is a weakly typed programming language because types are implicitly converted. It is an inferred typed language because variables can be assigned without declaring their type, except if they are to be treated as symbolic objects, and that their type can change. Values can come from constants, from computation involving values of other variables, or from the output of a function. For example:

>> x = 17
x =
 17

>> x = 'hat'
x =
hat

>> x = [3*4, pi/2]
x =
   12.0000    1.5708

>> y = 3*sin(x)
y =
   -1.6097    3.0000

Vectors and matrices

A simple array is defined using the colon syntax: initial:increment:terminator. For instance:

>> array = 1:2:9
array=
 1 3 5 7 9

defines a variable named array (or assigns a new value to an existing variable with the name array) which is an array consisting of the values 1, 3, 5, 7, and 9. That is, the array starts at 1 (the initial value), increments with each step from the previous value by 2 (the increment value), and stops once it reaches (or to avoid exceeding) 9 (the terminator value).

>> array = 1:3:9
array =
 1 4 7

the increment value can actually be left out of this syntax (along with one of the colons), to use a default value of 1.

>> ari = 1:5
ari =
 1 2 3 4 5

assigns to the variable named ari an array with the values 1, 2, 3, 4, and 5, since the default value of 1 is used as the incrementer.

Indexing is one-based, which is the usual convention for matrices in mathematics, although not for some programming languages such as C, C++, and Java.

Matrices can be defined by separating the elements of a row with blank space or comma and using a semicolon to terminate each row. The list of elements should be surrounded by square brackets: []. Parentheses: () are used to access elements and subarrays (they are also used to denote a function argument list).

>> A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]
A =
 16  3  2 13
  5 10 11  8
  9  6  7 12
  4 15 14  1

>> A(2,3)
ans =
 11

Sets of indices can be specified by expressions such as "2:4", which evaluates to [2, 3, 4]. For example, a submatrix taken from rows 2 through 4 and columns 3 through 4 can be written as:

>> A(2:4,3:4)
ans =
 11 8
 7 12
 14 1

A square identity matrix of size n can be generated using the function eye, and matrices of any size with zeros or ones can be generated with the functions zeros and ones, respectively.

>> eye(3,3)
ans =
 1 0 0
 0 1 0
 0 0 1

>> zeros(2,3)
ans =
 0 0 0
 0 0 0

>> ones(2,3)
ans =
 1 1 1
 1 1 1

Transposing a vector or a matrix is done either by the function transpose or by adding prime after a dot to the matrix. Without the dot MATLAB will perform conjugate transpose.

>> A = [1 ; 2],  B = A.', C = transpose(A)
A =
     1
     2
B =
     1     2
C =
     1     2

>> D = [0 3 ; 1 5], D.'
D =
     0     3
     1     5
ans =
     0     1
     3     5

Most MATLAB functions can accept matrices and will apply themselves to each element. For example, mod(2*J,n) will multiply every element in "J" by 2, and then reduce each element modulo "n". MATLAB does include standard "for" and "while" loops, but (as in other similar applications such as R), using the vectorized notation often produces code that is faster to execute. This code, excerpted from the function magic.m, creates a magic square M for odd values of n (MATLAB function meshgrid is used here to generate square matrices I and J containing 1:n).

[J,I] = meshgrid(1:n);
A = mod(I + J - (n + 3) / 2, n);
B = mod(I + 2 * J - 2, n);
M = n * A + B + 1;

Structures

MATLAB has structure data types. Since all variables in MATLAB are arrays, a more adequate name is "structure array", where each element of the array has the same field names. In addition, MATLAB supports dynamic field names (field look-ups by name, field manipulations, etc.). Unfortunately, MATLAB JIT does not support MATLAB structures, therefore just a simple bundling of various variables into a structure will come at a cost.

Functions

When creating a MATLAB function, the name of the file should match the name of the first function in the file. Valid function names begin with an alphabetic character, and can contain letters, numbers, or underscores. Functions are often case sensitive.

Function handles

MATLAB supports elements of lambda calculus by introducing function handles, or function references, which are implemented either in .m files or anonymous/nested functions.

Classes and object-oriented programming

MATLAB supports object-oriented programming including classes, inheritance, virtual dispatch, packages, pass-by-value semantics, and pass-by-reference semantics. However, the syntax and calling conventions are significantly different from other languages. MATLAB has value classes and reference classes, depending on whether the class has handle as a super-class (for reference classes) or not (for value classes).

Method call behavior is different between value and reference classes. For example, a call to a method

object.method();

can alter any member of object only if object is an instance of a reference class.

An example of a simple class is provided below.

classdef hello
    methods
        function greet(this)
            disp('Hello!')
        end
    end
end

When put into a file named hello.m, this can be executed with the following commands:

>> x = hello;
>> x.greet();
Hello!

Graphics and graphical user interface programming

MATLAB supports developing applications with graphical user interface (GUI) features. MATLAB includes GUIDE (GUI development environment) for graphically designing GUIs. It also has tightly integrated graph-plotting features. For example, the function plot can be used to produce a graph from two vectors x and y. The code:

x = 0:pi/100:2*pi;
y = sin(x);
plot(x,y)

produces the following figure of the sine function:



A MATLAB program can produce three-dimensional graphics using the functions surf, plot3 or mesh.

In MATLAB, graphical user interfaces can be programmed with the GUI design environment (GUIDE) tool.

Interfacing with other languages

MATLAB can call functions and subroutines written in the programming languages C or Fortran. A wrapper function is created allowing MATLAB data types to be passed and returned. MEX files (MATLAB executables) are the dynamically loadable object files created by compiling such functions. Since 2014 increasing two-way interfacing with Python was being added.

Libraries written in Perl, Java, ActiveX or .NET can be directly called from MATLAB, and many MATLAB libraries (for example XML or SQL support) are implemented as wrappers around Java or ActiveX libraries. Calling MATLAB from Java is more complicated, but can be done with a MATLAB toolbox which is sold separately by MathWorks, or using an undocumented mechanism called JMI (Java-to-MATLAB Interface), (which should not be confused with the unrelated Java Metadata Interface that is also called JMI). Official MATLAB API for Java was added in 2016.
As alternatives to the MuPAD based Symbolic Math Toolbox available from MathWorks, MATLAB can be connected to Maple or Mathematica.

Libraries also exist to import and export MathML.

License

MATLAB is a proprietary product of MathWorks, so users are subject to vendor lock-in. Although MATLAB Builder products can deploy MATLAB functions as library files which can be used with .NET or Java application building environment, future development will still be tied to the MATLAB language.

Each toolbox is purchased separately. If an evaluation license is requested, the MathWorks sales department requires detailed information about the project for which MATLAB is to be evaluated. If granted (which it often is), the evaluation license is valid for two to four weeks. A student version of MATLAB is available as is a home-use license for MATLAB, Simulink, and a subset of Mathwork's Toolboxes at substantially reduced prices.

It has been reported that European Union (EU) competition regulators are investigating whether MathWorks refused to sell licenses to a competitor. The regulators dropped the investigation after the complainant withdrew its accusation and no evidence of wrongdoing was found.

Alternatives

See also: list of numerical analysis software and comparison of numerical analysis software

MATLAB has a number of competitors. Commercial competitors include Mathematica, TK Solver, Maple, and IDL. There are also free open source alternatives to MATLAB, in particular GNU Octave, Scilab, FreeMat, and SageMath, which are intended to be mostly compatible with the MATLAB language; the Julia programming language also initially used MATLAB-like syntax. Among other languages that treat arrays as basic entities (array programming languages) are APL, Fortran 90 and higher, S-Lang, as well as the statistical languages R and S. There are also libraries to add similar functionality to existing languages, such as IT++ for C++, Perl Data Language for Perl, ILNumerics for .NET, NumPy/SciPy/matplotlib for Python, SciLua/Torch for Lua, SciRuby for Ruby, and Numeric.js for JavaScript.

GNU Octave is unique from other alternatives because it treats incompatibility with MATLAB as a bug (see MATLAB Compatibility of GNU Octave), therefore, making GNU Octave a superset of the MATLAB language.

Release history

Version	Release name	Number	Bundled JVM	Year	Release date	Notes
MATLAB 1.0				1984
MATLAB 2				1986
MATLAB 3				1987
MATLAB 3.5				1990		Ran on DOS but needed at least a 386 processor; version 3.5m needed math coprocessor
MATLAB 4				1992		Ran on Windows 3.1x and Macintosh
MATLAB 4.2c				1994		Ran on Windows 3.1x, needed a math coprocessor
MATLAB 5.0	Volume 8			1996	December 1996	Unified releases across all platforms
MATLAB 5.1	Volume 9			1997	May 1997
MATLAB 5.1.1	R9.1
MATLAB 5.2	R10			1998	March 1998	Last version working on classic Macs
MATLAB 5.2.1	R10.1
MATLAB 5.3	R11			1999	January 1999
MATLAB 5.3.1	R11.1				November 1999
MATLAB 6.0	R12	12	1.1.8	2000	November 2000	First release with bundled Java virtual machine (JVM)
MATLAB 6.1	R12.1		1.3.0	2001	June 2001
MATLAB 6.5	R13	13	1.3.1	2002	July 2002
MATLAB 6.5.1	R13SP1			2003
MATLAB 6.5.2	R13SP2					Last release for IBM/AIX, Alpha/TRU64, and SGI/IRIX
MATLAB 7	R14	14	1.4.2	2004	June 2004	Introduced anonymous and nested functions Re-introduced for Mac (under Mac OS X)
MATLAB 7.0.1	R14SP1				October 2004
MATLAB 7.0.4	R14SP2		1.5.0	2005	March 7, 2005	Support for memory-mapped files
MATLAB 7.1	R14SP3		1.5.0		September 1, 2005
MATLAB 7.2	R2006a	15	1.5.0	2006	March 1, 2006
MATLAB 7.3	R2006b	16	1.5.0		September 1, 2006	HDF5-based MAT-file support
MATLAB 7.4	R2007a	17	1.5.0_07	2007	March 1, 2007	New bsxfun function to apply element-by-element binary operation with singleton expansion enabled
MATLAB 7.5	R2007b	18	1.6.0		September 1, 2007	Last release for Windows 2000 and PowerPC Mac; License Server support for Windows Vista; new internal format for P-code
MATLAB 7.6	R2008a	19	1.6.0	2008	March 1, 2008	Major enhancements to object-oriented programming abilities with a new class definition syntax, and ability to manage namespaces with packages
MATLAB 7.7	R2008b	20	1.6.0_04		October 9, 2008	New Map data structure: upgrades to random number generators
MATLAB 7.8	R2009a	21	1.6.0_04	2009	March 6, 2009	First release for Microsoft 32-bit & 64-bit Windows 7, new external interface to .NET Framework
MATLAB 7.9	R2009b	22	1.6.0_12		September 4, 2009	First release for Intel 64-bit Mac, and last for Solaris SPARC; new use for the tilde operator (~) to ignore arguments in function calls
MATLAB 7.9.1	R2009bSP1		1.6.0_12	2010	April 1, 2010	bug fixes.
MATLAB 7.10	R2010a	23	1.6.0_12		March 5, 2010	Last release for Intel 32-bit Mac
MATLAB 7.11	R2010b	24	1.6.0_17		September 3, 2010	Add support for enumerations
MATLAB 7.11.1	R2010bSP1		1.6.0_17	2011	March 17, 2011	bug fixes and updates
MATLAB 7.11.2	R2010bSP2		1.6.0_17		April 5, 2012	bug fixes
MATLAB 7.12	R2011a	25	1.6.0_17		April 8, 2011	New rng function to control random number generation
MATLAB 7.13	R2011b	26	1.6.0_17		September 1, 2011	Access-change parts of variables directly in MAT-files, without loading into memory; increased maximum local workers with Parallel Computing Toolbox from 8 to 12
MATLAB 7.14	R2012a	27	1.6.0_17	2012	March 1, 2012	Last version with 32-bit Linux support.
MATLAB 8	R2012b	28	1.6.0_17		September 11, 2012	First release with Toolstrip interface; MATLAB Apps. redesigned documentation system
MATLAB 8.1	R2013a	29	1.6.0_17	2013	March 7, 2013	New unit testing framework
MATLAB 8.2	R2013b	30	1.7.0_11		September 6, 2013	Built in Java Runtime Environment (JRE) updated to version 7; New table data type
MATLAB 8.3	R2014a	31	1.7.0_11	2014	March 7, 2014	Simplified compiler setup for building MEX-files; USB Webcams support in core MATLAB; number of local workers no longer limited to 12 with Parallel Computing Toolbox
MATLAB 8.4	R2014b	32	1.7.0_11		October 3, 2014	New class-based graphics engine (a.k.a. HG2); tabbing function in GUI; improved user toolbox packaging and help files; new objects for time-date manipulations; Git-Subversion integration in IDE; big data abilities with MapReduce (scalable to Hadoop); new py package for using Python from inside MATLAB, new engine interface to call MATLAB from Python; several new and improved functions: webread (RESTful web services with JSON/XML support), tcpclient (socket-based connections), histcounts, histogram, animatedline, and others
MATLAB 8.5	R2015a	33	1.7.0_60	2015	March 5, 2015	Last release supporting Windows XP and Windows Vista
MATLAB 8.5	R2015aSP1		1.7.0_60		October 14, 2015
MATLAB 8.6	R2015b	34	1.7.0_60		September 3, 2015	New MATLAB execution engine (a.k.a. LXE); graph and digraph classes to work with graphs and networks; MinGW-w64 as supported compiler on Windows; Last version with 32-bit support
MATLAB 9.0	R2016a	35	1.7.0_60	2016	March 3, 2016	Live Scripts: interactive documents that combine text, code, and output (in the style of Literate programming); App Designer: a new development environment for building apps (with new kind of UI figures, axes, and components); pause execution of running programs using a Pause Button
MATLAB 9.1	R2016b	36	1.7.0_60		September 15, 2016	define local functions in scripts; automatic expansion of dimensions (previously provided via explicit call to bsxfun); tall arrays for Big data; new string type; new functions to encode/decode JSON; official MATLAB Engine API for Java
MATLAB 9.2	R2017a	37	1.7.0_60	2017	March 9, 2017	MATLAB Online: cloud-based MATLAB desktop accessed in a web browser; double-quoted strings; new memoize function for Memoization; expanded object properties validation; mocking framework for unit testing; MEX targets 64-bit by default; new heatmap function for creating heatmap charts
MATLAB 9.3	R2017b	38	1.8.0_121		September 21, 2017
MATLAB 9.4	R2018a	39	1.8.0_144	2018	March 15, 2018
MATLAB 9.5	R2018b	40	1.8.0_152		September 12, 2018

The number (or release number) is the version reported by Concurrent License Manager program FLEXlm.

For a complete list of changes of both MATLAB and official toolboxes, consult the MATLAB release notes.

File extensions

MATLAB

.m 

MATLAB code (function, script, or class)

.mat 

MATLAB data (binary file for storing variables)

.mex* (.mexw32, .mexw64, .mexglx, .mexa64, .mexmaci64, ...) 

MATLAB executable MEX-files (platform specific, e.g. ".mexmac" for the Mac, ".mexglx" for Linux, etc.)

.p 

MATLAB content-obscured .m file (P-code)

.mlx 

MATLAB live script

.fig 

MATLAB figures (created with GUIDE)

.mlapp 

MATLAB apps (created with App Designer)

.mlappinstall 

MATLAB packaged App Installer

.mlpkginstall

support package installer (add-on for third-party hardware)

.mltx, .mltbx

packaged custom toolbox

.prj

project file used by various solutions (packaged app/toolbox projects, MATLAB Compiler/Coder projects, Simulink projects)

.rpt

report setup file created by MATLAB Report Generator

Simulink

.mdl 

Simulink Model

.mdlp 

Simulink Protected Model

.slx 

Simulink Model (SLX format)

.slxp 

Simulink Protected Model (SLX format)

Simscape

.ssc 

Simscape Model

MuPAD

.mn 

MuPAD Notebook

.mu 

MuPAD Code

.xvc, .xvz 

MuPAD Graphics

Third-party

.jkt 

GPU Cache file generated by Jacket for MATLAB (AccelerEyes)

.mum 

MATLAB CAPE-OPEN Unit Operation Model File (AmsterCHEM)

Easter eggs

Several easter eggs exist in MATLAB. These include hidden pictures, and jokes. For example, typing in "spy" used to generate a picture of the spies from Spy vs Spy, but now displays an image of a dog. Typing in "why" randomly outputs a philosophical answer. Other commands include "penny", "toilet", "image", and "life". Not every Easter egg appears in every version of MATLAB.

Simulation-based optimization

From Wikipedia, the free encyclopedia

 

Simulation-based optimization integrates optimization techniques into simulation analysis. Because of the complexity of the simulation, the objective function may become difficult and expensive to evaluate.

Once a system is mathematically modeled, computer-based simulations provide information about its behavior. Parametric simulation methods can be used to improve the performance of a system. In this method, the input of each variable is varied with other parameters remaining constant and the effect on the design objective is observed. This is a time-consuming method and improves the performance partially. To obtain the optimal solution with minimum computation and time, the problem is solved iteratively where in each iteration the solution moves closer to the optimum solution. Such methods are known as ‘numerical optimization’ or ‘simulation-based optimization’.

In simulation experiment, the goal is to evaluate the effect of different values of input variables on a system. However, the interest is sometimes in finding the optimal value for input variables in terms of the system outcomes. One way could be running simulation experiments for all possible input variables. However, this approach is not always practical due to several possible situations and it just makes it intractable to run experiments for each scenario. For example, there might be too many possible values for input variables, or the simulation model might be too complicated and expensive to run for suboptimal input variable values. In these cases, the goal is to find optimal values for the input variables rather than trying all possible values. This process is called simulation optimization.

Specific simulation–based optimization methods can be chosen according to figure 1 based on the decision variable types.

Fig.1 Classification of simulation based optimization according to variable types

Optimization exists in two main branches of operational research:

Optimization parametric (static) – The objective is to find the values of the parameters, which are “static” for all states, with the goal of maximizing or minimizing a function. In this case, one can use mathematical programming, such as linear programming. In this scenario, simulation helps when the parameters contain noise or the evaluation of the problem would demand excessive computer time, due to its complexity.

Optimization control (dynamic) – This is used largely in computer science and electrical engineering. The optimal control is per state and the results change in each of them. One can use mathematical programming, as well as dynamic programming. In this scenario, simulation can generate random samples and solve complex and large-scale problems.

Simulation-based optimization methods

The main approaches in simulation optimization are discussed below. 

Statistical ranking and selection methods (R/S)

Ranking and selection methods are designed for problems where the alternatives are fixed and known, and simulation is used to estimate the system performance. In the simulation optimization setting, applicable methods include indifference zone approaches, optimal computing budget allocation, and knowledge gradient algorithms.

Response surface methodology (RSM)

In response surface methodology, the objective is to find the relationship between the input variables and the response variables. The process starts from trying to fit a linear regression model. If the P-value turns out to be low, then a higher degree polynomial regression, which is usually quadratic, will be implemented. The process of finding a good relationship between input and response variables will be done for each simulation test. In simulation optimization, response surface method can be used to find the best input variables that produce desired outcomes in terms of response variables.

Heuristic methods

Heuristic methods change accuracy by speed. Their goal is to find a good solution faster than the traditional methods, when they are too slow or fail in solving the problem. Usually they find local optimal instead of the optimal value; however, the values are considered close enough of the final solution. Examples of this kind of method is tabu search or Genetic algorithm.

Metamodels enable researchers to obtain reliable approximate model outputs without running expensive and time-consuming computer simulations. Therefore, the process of model optimization can take less computation time and cost. 

Stochastic approximation

Stochastic approximation is used when the function cannot be computed directly, only estimated via noisy observations. In this scenarios, this method (or family of methods) looks for the extrema of these function. The objective function would be:

${\displaystyle {\underset {{\text{x}}\in \theta }{\min }}f{\bigl (}{\text{x}}{\bigr )}={\underset {{\text{x}}\in \theta }{\min }}\mathrm {E} [F{\bigl (}{\text{x,y}})]}$

${\displaystyle y}$ is a random variable that represents the noise.

${\displaystyle x}$ is the parameter that minimizes ${\displaystyle f{\bigl (}{\text{x}}{\bigr )}}$.

${\displaystyle \theta }$ is the domain of the parameter ${\displaystyle x}$.

Derivative-free optimization methods

Derivative-free optimization is a subject of mathematical optimization. This method is applied to a certain optimization problem when its derivatives are unavailable or unreliable. Derivative-free methods establish a model based on sample function values or directly draw a sample set of function values without exploiting a detailed model. Since it needs no derivatives, it cannot be compared to derivative-based methods.

For unconstrained optimization problems, it has the form:

${\displaystyle {\underset {{\text{x}}\in \mathbb {R} ^{n}}{\min }}f{\bigl (}{\text{x}}{\bigr )}}$

The limitations of derivative-free optimization:

• It is usually cannot handle optimization problems with a few tens of variables; the results via this method are usually not so accurate;
• When confronted with minimizing non-convex functions, it will show its limitation;
• Derivative-free optimization methods are simple and easy; however, they are not so good in theory and in practice.

Dynamic programming and neuro-dynamic programming

Dynamic programming

Dynamic programming deals with situations where decisions are made in stages. The key to this kind of problems is to trade off the present and future costs.

One dynamic basic model has two features:

1. It has a discrete time dynamic system;
2. The cost function is additive over time.

For discrete features, dynamic programming has the form:

${\displaystyle x_{k+1}=f_{k}(x_{k},u_{k},w_{k}),k=0,1,...,N-1}$

${\displaystyle k}$ represents the index of discrete time.

${\displaystyle x_{k}}$ is the state of the time k, it contains the past information and prepare it for the future optimization.

${\displaystyle u_{k}}$ is the control variable.

${\displaystyle w_{k}}$ is the random parameter.

For the cost function, it has the form:

${\displaystyle g_{N}(X_{N})+\sum _{k=0}^{N-1}gk(x_{k},u_{k},W_{k})}$

${\displaystyle g_{N}(X_{N})}$ is the cost at the end of the process.

As the cost cannot be optimized meaningfully, it can be used the expect value:

${\displaystyle E\{g_{N}(X_{N})+\sum _{k=0}^{N-1}g_{k}(x_{k},u_{k},W_{k})\}}$

Neuro-dynamic programming

Neuro-dynamic programming is the same as dynamic programming except that the former has the concept of approximation architectures. It combines artificial intelligence, simulation-base algorithms, and functional approach techniques. “Neuro” in this term origins from artificial intelligence community. It means learning how to make improved decisions for the future via built-in mechanism based on the current behavior. The most important part of neuro-dynamic programming is to build a trained neuro network for the optimal problem.

Limitations

Simulation based optimization has some limitations, such as the difficulty of creating a model that imitates the dynamic behavior of a system in a way that is considered good enough for its representation. Another problem is complexity in the determining uncontrollable parameters of both real-world system and simulation. Moreover, only a statistical estimation of real values can be obtained. It is not easy to determine the objective function, since it is a result of measurements, which can be harmful for the solutions.