Diesel fuel

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Diesel_fuel 

A tank of diesel fuel on a truck

Diesel fuel /ˈdiːzəl/, also called diesel oil, is any liquid fuel specifically designed for use in a diesel engine, a type of internal combustion engine in which fuel ignition takes place without a spark as a result of compression of the inlet air and then injection of fuel. Therefore, diesel fuel needs good compression ignition characteristics.

The most common type of diesel fuel is a specific fractional distillate of petroleum fuel oil, but alternatives that are not derived from petroleum, such as biodiesel, biomass to liquid (BTL) or gas to liquid (GTL) diesel are increasingly being developed and adopted. To distinguish these types, petroleum-derived diesel is sometimes called petrodiesel in some academic circles.

In many countries, diesel fuel is standardised. For example, in the European Union, the standard for diesel fuel is EN 590. Diesel fuel has many colloquial names; most commonly, it is simply referred to as diesel. In the UK, diesel fuel for on-road use is commonly abbreviated DERV, standing for diesel-engined road vehicle, which carries a tax premium over equivalent fuel for non-road use. In Australia, diesel fuel is also known as distillate, and in Indonesia, it is known as Solar, a trademarked name from the country's national petroleum company Pertamina.

Ultra-low-sulfur diesel (ULSD) is a diesel fuel with substantially lowered sulfur contents. As of 2016, almost all of the petroleum-based diesel fuel available in the UK, mainland Europe, and North America is of a ULSD type.

Before diesel fuel had been standardised, the majority of diesel engines typically ran on cheap fuel oils. These fuel oils are still used in watercraft diesel engines. Despite being specifically designed for diesel engines, diesel fuel can also be used as fuel for several non-diesel engines, for example the Akroyd engine, the Stirling engine, or boilers for steam engines.

History

Origins

Diesel fuel originated from experiments conducted by German scientist and inventor Rudolf Diesel for his compression-ignition engine he invented in 1892. Originally, Diesel did not consider using any specific type of fuel, instead, he claimed that the operating principle of his rational heat motor would work with any kind of fuel in any state of matter. However, both the first diesel engine prototype and the first functional Diesel engine were only designed for liquid fuels.

At first, Diesel tested crude oil from Pechelbronn, but soon replaced it with petrol and kerosene, because crude oil proved to be too viscous, with the main testing fuel for the Diesel engine being kerosene. In addition to that, Diesel experimented with different types of lamp oil from various sources, as well as different types of petrol and ligroin, which all worked well as Diesel engine fuels. Later, Diesel tested coal tar creosote, paraffin oil, crude oil, gasoil, and fuel oil, which eventually worked as well. In Scotland and France, shale oil was used as fuel for the first 1898 production Diesel engines because other fuels were too expensive. In 1900, the French Otto society built a Diesel engine for the use with crude oil, which was exhibited at the 1900 Paris Exposition and the 1911 World's Fair in Paris. The engine actually ran on peanut oil instead of crude oil, and no modifications were necessary for peanut oil operation.

During his first Diesel engine tests, Diesel also used illuminating gas as fuel, and managed to build functional designs, both with and without pilot injection. According to Diesel, neither was a coal-dust–producing industry existent, nor was fine, high quality coal-dust commercially available in the late 1890s. This is the reason why the Diesel engine was never designed or planned as a coal-dust engine. Only in December 1899, did Diesel test a coal-dust prototype, which used external mixture formation and liquid fuel pilot injection. This engine proved to be functional, but suffered from piston ring failure after a very few minutes due to coal dust deposition.

Since the 20th century

Before diesel fuel had been standardised, diesel engines typically ran on cheap fuel oils. In the United States, these were distilled from petroleum, whereas in Europe, coal-tar creosote oil was used. Some diesel engines were fuelled with mixtures of several different fuels, such as petrol, kerosine, rapeseed oil, or lubricating oil, because they were untaxed and thus cheap. The introduction of motor-vehicle diesel engines, such as the Mercedes-Benz OM 138, in the 1930s meant that higher quality fuels with proper ignition characteristics were needed. At first no improvements were made to motor-vehicle diesel fuel quality. After World War II, the first modern high quality diesel fuels were standardised. These standards were, for instance, the DIN 51601, VTL 9140-001, and NATO F 54 standards. In 1993, the DIN 51601 was rendered obsolete by the new EN 590 standard, which has been used in the European Union ever since. In sea-going watercraft, where diesel propulsion had gained prevalence by the late 1970s due to increasing fuel costs caused by the 1970s energy crisis, cheap heavy fuel oils are still used instead of conventional motor-vehicle diesel fuel. These heavy fuel oils (often called Bunker C) can be used in diesel-powered and steam-powered vessels.

Types

Diesel fuel is produced from various sources, the most common being petroleum. Other sources include biomass, animal fat, biogas, natural gas, and coal liquefaction.

Petroleum diesel

A modern diesel dispenser

Petroleum diesel, also called petrodiesel, fossil diesel, or mineral diesel, is the most common type of diesel fuel. It is produced from the fractional distillation of crude oil between 200 and 350 °C (392 and 662 °F) at atmospheric pressure, resulting in a mixture of carbon chains that typically contain between 9 and 25 carbon atoms per molecule.

Synthetic diesel

Main article: Synthetic fuel

See also: Electrofuel

Synthetic diesel can be produced from any carbonaceous material, including biomass, biogas, natural gas, coal and many others. The raw material is gasified into synthesis gas, which after purification is converted by the Fischer–Tropsch process to a synthetic diesel.

The process is typically referred to as biomass-to-liquid (BTL), gas-to-liquid (GTL) or coal-to-liquid (CTL), depending on the raw material used.

Paraffinic synthetic diesel generally has a near-zero content of sulfur and very low aromatics content, reducing unregulated emissions of toxic hydrocarbons, nitrous oxides and particulate matter (PM).

Biodiesel

Main article: Biodiesel

 

Biodiesel made from soybean oil

Biodiesel is obtained from vegetable oil or animal fats (biolipids) which are mainly fatty acid methyl esters (FAME), and transesterified with methanol. It can be produced from many types of oils, the most common being rapeseed oil (rapeseed methyl ester, RME) in Europe and soybean oil (soy methyl ester, SME) in the US. Methanol can also be replaced with ethanol for the transesterification process, which results in the production of ethyl esters. The transesterification processes use catalysts, such as sodium or potassium hydroxide, to convert vegetable oil and methanol into biodiesel and the undesirable byproducts glycerine and water, which will need to be removed from the fuel along with methanol traces. Biodiesel can be used pure (B100) in engines where the manufacturer approves such use, but it is more often used as a mix with diesel, BXX where XX is the biodiesel content in percent.

FAME used as fuel is specified in DIN EN 14214 and ASTM D6751 standards.

Fuel Injection Equipment (FIE) manufacturers have raised several concerns regarding biodiesel, identifying FAME as being the cause of the following problems: corrosion of fuel injection components, low-pressure fuel system blockage, increased dilution and polymerization of engine sump oil, pump seizures due to high fuel viscosity at low temperature, increased injection pressure, elastomeric seal failures and fuel injector spray blockage. Pure biodiesel has an energy content about 5–10% lower than petroleum diesel. The loss in power when using pure biodiesel is 5–7%.

Unsaturated fatty acids are the source for the lower oxidation stability. They react with oxygen and form peroxides and result in degradation byproducts, which can cause sludge and lacquer in the fuel system.

As biodiesel contains low levels of sulfur, the emissions of sulfur oxides and sulfates, major components of acid rain, are low. Use of biodiesel also results in reductions of unburned hydrocarbons, carbon monoxide (CO), and particulate matter. CO emissions using biodiesel are substantially reduced, on the order of 50% compared to most petrodiesel fuels. The exhaust emissions of particulate matter from biodiesel have been found to be 30% lower than overall particulate matter emissions from petrodiesel. The exhaust emissions of total hydrocarbons (a contributing factor in the localized formation of smog and ozone) are up to 93% lower for biodiesel than diesel fuel.

Biodiesel also may reduce health risks associated with petroleum diesel. Biodiesel emissions showed decreased levels of polycyclic aromatic hydrocarbon (PAH) and nitrated PAH compounds, which have been identified as potential carcinogens. In recent testing, PAH compounds were reduced by 75–85%, except for benz(a)anthracene, which was reduced by roughly 50%. Targeted nPAH compounds were also reduced dramatically with biodiesel fuel, with 2-nitrofluorene and 1-nitropyrene reduced by 90%, and the rest of the nPAH compounds reduced to only trace levels.

Hydrogenated oils and fats

This category of diesel fuels involves converting the triglycerides in vegetable oil and animal fats into alkanes by refining and hydrogenation, such as Neste Renewable Diesel or H-Bio. The produced fuel has many properties that are similar to synthetic diesel, and are free from the many disadvantages of FAME.

DME

Dimethyl ether, DME, is a synthetic, gaseous diesel fuel that results in clean combustion with very little soot and reduced NOx emissions.

Storage

Large diesel fuel tanks in Sörnäinen, Helsinki, Finland

In the US, diesel is recommended to be stored in a yellow container to differentiate it from kerosene, which is typically kept in blue containers, and gasoline (petrol), which is typically kept in red containers. In the UK, diesel is normally stored in a black container to differentiate it from unleaded or leaded petrol, which are stored in green and red containers, respectively.

Standards

The diesel engine is a multifuel engine and can run on a huge variety of fuels. However, development of high-performance, high-speed diesel engines for cars and lorries in the 1930s meant that a proper fuel specifically designed for such engines was needed: diesel fuel. In order to ensure consistent quality, diesel fuel is standardised; the first standards were introduced after World War II. Typically, a standard defines certain properties of the fuel, such as cetane number, density, flash point, sulphur content, or biodiesel content. Diesel fuel standards include:

Diesel fuel

• EN 590 (European Union)
• ASTM D975 (United States)
• GOST R 52368 (Russia; equivalent to EN 590)
• NATO F 54 (NATO; equivalent to EN 590)
• DIN 51601 (West Germany; obsolete)

Biodiesel fuel

• EN 14214 (European Union)
• ASTM D6751 (United States)
• CAN/CGSB-3.524 (Canada)

Measurements and pricing

Cetane number

Main article: Cetane number

The principal measure of diesel fuel quality is its cetane number. A cetane number is a measure of the delay of ignition of a diesel fuel. A higher cetane number indicates that the fuel ignites more readily when sprayed into hot compressed air. European (EN 590 standard) road diesel has a minimum cetane number of 51. Fuels with higher cetane numbers, normally "premium" diesel fuels with additional cleaning agents and some synthetic content, are available in some markets.

Fuel value and price

Further information: Gasoline and diesel usage and pricing

About 86.1% of diesel fuel mass is carbon, and when burned, it offers a net heating value of 43.1 MJ/kg as opposed to 43.2 MJ/kg for gasoline. Due to the higher density, diesel fuel offers a higher volumetric energy density: the density of EN 590 diesel fuel is defined as 0.820 to 0.845 kg/L (6.84 to 7.05 lb/US gal) at 15 °C (59 °F), about 9.0-13.9% more than EN 228 gasoline (petrol)'s 0.720–0.775 kg/L (6.01–6.47 lb/US gal) at 15 °C, which should be put into consideration when comparing volumetric fuel prices. The CO₂ emissions from diesel are 73.25 g/MJ, just slightly lower than for gasoline at 73.38 g/MJ.

Diesel fuel is generally simpler to refine from petroleum than gasoline, and contains hydrocarbons having a boiling point in the range of 180–360 °C (356–680 °F). Additional refining is required to remove sulfur, which contributes to a sometimes higher cost. In many parts of the United States and throughout the United Kingdom and Australia, diesel fuel may be priced higher than petrol. Reasons for higher-priced diesel include the shutdown of some refineries in the Gulf of Mexico, diversion of mass refining capacity to gasoline production, and a recent transfer to ultra-low-sulfur diesel (ULSD), which causes infrastructural complications. In Sweden, a diesel fuel designated as MK-1 (class 1 environmental diesel) is also being sold. This is a ULSD that also has a lower aromatics content, with a limit of 5%. This fuel is slightly more expensive to produce than regular ULSD. In Germany, the fuel tax on diesel fuel is about 28% lower than the petrol fuel tax.

Taxation

Diesel fuel is similar to heating oil, which is used in central heating. In Europe, the United States, and Canada, taxes on diesel fuel are higher than on heating oil due to the fuel tax, and in those areas, heating oil is marked with fuel dyes and trace chemicals to prevent and detect tax fraud. "Untaxed" diesel (sometimes called "off-road diesel" or "red diesel" due to its red dye) is available in some countries for use primarily in agricultural applications, such as fuel for tractors, recreational and utility vehicles or other noncommercial vehicles that do not use public roads. This fuel may have sulfur levels that exceed the limits for road use in some countries (e.g. US).

This untaxed diesel is dyed red for identification, and using this untaxed diesel fuel for a typically taxed purpose (such as driving use), the user can be fined (e.g. US$10,000 in the US). In the United Kingdom, Belgium and the Netherlands, it is known as red diesel (or gas oil), and is also used in agricultural vehicles, home heating tanks, refrigeration units on vans/trucks which contain perishable items such as food and medicine and for marine craft. Diesel fuel, or marked gas oil is dyed green in the Republic of Ireland and Norway. The term "diesel-engined road vehicle" (DERV) is used in the UK as a synonym for unmarked road diesel fuel. In India, taxes on diesel fuel are lower than on petrol, as the majority of the transportation for grain and other essential commodities across the country runs on diesel.

Taxes on biodiesel in the US vary between states. Some states (Texas, for example) have no tax on biodiesel and a reduced tax on biodiesel blends equivalent to the amount of biodiesel in the blend, so that B20 fuel is taxed 20% less than pure petrodiesel. Other states, such as North Carolina, tax biodiesel (in any blended configuration) the same as petrodiesel, although they have introduced new incentives to producers and users of all biofuels.

Uses

Diesel fuel is mostly used in high-speed diesel engines, especially motor-vehicle (e.g. car, lorry) diesel engines, but not all diesel engines run on diesel fuel. For example, large two-stroke watercraft engines typically use heavy fuel oils instead of diesel fuel, and certain types of diesel engines, such as MAN M-System engines, are designed to run on petrol with knock resistances of up to 86 RON. On the other hand, gas turbine and some other types of internal combustion engines, and external combustion engines, can also be designed to take diesel fuel.

The viscosity requirement of diesel fuel is usually specified at 40 °C. A disadvantage of diesel fuel in cold climates is that its viscosity increases as the temperature decreases, changing it into a gel (see Compression Ignition – Gelling) that cannot flow in fuel systems. Special low-temperature diesel contains additives to keep it liquid at lower temperatures.

On-road vehicles

Trucks and buses, which were often otto-powered in the 1920s through 1950s, are now almost exclusively diesel-powered. Due to its ignition characteristics, diesel fuel is thus widely used in these vehicles. Since diesel fuel is not well-suited for otto engines, passenger cars, which often use otto or otto-derived engines, typically run on petrol instead of diesel fuel. However, especially in Europe and India, many passenger cars have, due to better engine efficiency, diesel engines, and thus run on regular diesel fuel.

Railroad

See also: Dieselization and Diesel locomotive

Diesel displaced coal and fuel oil for steam-powered vehicles in the latter half of the 20th century, and is now used almost exclusively for the combustion engines of self-powered rail vehicles (locomotives and railcars).

Aircraft

Packard DR-980 9-cylinder diesel aircraft engine, used in the first diesel-engine airplane

In general, diesel engines are not well-suited for planes and helicopters. This is because of the diesel engine's comparatively low power-to-mass ratio, meaning that diesel engines are typically rather heavy, which is a disadvantage in aircraft. Therefore, there is little need for using diesel fuel in aircraft, and diesel fuel is not commercially used as aviation fuel. Instead, petrol (Avgas), and jet fuel (e. g. Jet A-1) are used. However, especially in the 1920s and 1930s, numerous series-production aircraft diesel engines that ran on fuel oils were made, because they had several advantages: their fuel consumption was low, they were reliable, not prone to catching fire, and required minimal maintenance. The introduction of petrol direct injection in the 1930s outweighed these advantages, and aircraft diesel engines quickly fell out of use. With improvements in power-to-mass ratios of diesel engines, several on-road diesel engines have been converted to and certified for aircraft use since the early 21st century. These engines typically run on Jet A-1 aircraft fuel (but can also run on diesel fuel). Jet A-1 has ignition characteristics similar to diesel fuel, and is thus suited for certain (but not all) diesel engines.

Military vehicles

Until World War II, several military vehicles, especially those that required high engine performance (armored fighting vehicles, for example the M26 Pershing or Panther tanks), used conventional otto engines and ran on petrol. Ever since World War II, several military vehicles with diesel engines have been made, capable of running on diesel fuel. This is because diesel engines are more fuel efficient, and diesel fuel is less prone to catching fire. Some of these diesel-powered vehicles (such as the Leopard 1 or MAN 630) still ran on petrol, and some military vehicles were still made with otto engines (e. g. Ural-375 or Unimog 404), incapable of running on diesel fuel.

Tractors and heavy equipment

Today's tractors and heavy equipment are mostly diesel-powered. Among tractors, only the smaller classes may also offer gasoline-fuelled engines. The dieselization of tractors and heavy equipment began in Germany before World War II but was unusual in the United States until after that war. During the 1950s and 1960s, it progressed in the US as well. Diesel fuel is commonly used in oil and gas extracting equipment, although some locales use electric or natural gas powered equipment.

Tractors and heavy equipment were often multifuel in the 1920s through 1940s, running either spark-ignition and low-compression engines, akryod engines, or diesel engines. Thus many farm tractors of the era could burn gasoline, alcohol, kerosene, and any light grade of fuel oil such as heating oil, or tractor vaporising oil, according to whichever was most affordable in a region at any given time. On US farms during this era, the name "distillate" often referred to any of the aforementioned light fuel oils. Spark ignition engines did not start as well on distillate, so typically a small auxiliary gasoline tank was used for cold starting, and the fuel valves were adjusted several minutes later, after warm-up, to transition to distillate. Engine accessories such as vaporizers and radiator shrouds were also used, both with the aim of capturing heat, because when such an engine was run on distillate, it ran better when both it and the air it inhaled were warmer rather than at ambient temperature. Dieselization with dedicated diesel engines (high-compression with mechanical fuel injection and compression ignition) replaced such systems and made more efficient use of the diesel fuel being burned.

Other uses

Poor quality diesel fuel has been used as an extraction agent for liquid–liquid extraction of palladium from nitric acid mixtures. Such use has been proposed as a means of separating the fission product palladium from PUREX raffinate which comes from used nuclear fuel. In this system of solvent extraction, the hydrocarbons of the diesel act as the diluent while the dialkyl sulfides act as the extractant. This extraction operates by a solvation mechanism. So far, neither a pilot plant nor full scale plant has been constructed to recover palladium, rhodium or ruthenium from nuclear wastes created by the use of nuclear fuel.

Diesel fuel is often used as the main ingredient in oil-base mud drilling fluid. The advantage of using diesel is its low cost and its ability to drill a wide variety of difficult strata, including shale, salt and gypsum formations. Diesel-oil mud is typically mixed with up to 40% brine water. Due to health, safety and environmental concerns, Diesel-oil mud is often replaced with vegetable, mineral, or synthetic food-grade oil-base drilling fluids, although diesel-oil mud is still in widespread use in certain regions.

During development of rocket engines in Germany during World War II J-2 Diesel fuel was used as the fuel component in several engines including the BMW 109-718. J-2 diesel fuel was also used as a fuel for gas turbine engines.

Chemical analysis

Chemical composition

Diesel does not mix with water.

In the United States, petroleum-derived diesel is composed of about 75% saturated hydrocarbons (primarily paraffins including n, iso, and cycloparaffins), and 25% aromatic hydrocarbons (including naphthalenes and alkylbenzenes). The average chemical formula for common diesel fuel is C₁₂H₂₃, ranging approximately from C₁₀H₂₀ to C₁₅H₂₈.

Chemical properties

Further information: Diesel exhaust and Gel point (petroleum)

Most diesel fuels freeze at common winter temperatures, while the temperatures greatly vary. Petrodiesel typically freezes around temperatures of −8.1 °C (17.5 °F), whereas biodiesel freezes between temperatures of 2° to 15 °C (35° to 60 °F). The viscosity of diesel noticeably increases as the temperature decreases, changing it into a gel at temperatures of −19 °C (−2.2 °F) to −15 °C (5 °F), that cannot flow in fuel systems. Conventional diesel fuels vaporise at temperatures between 149 °C and 371 °C.

Conventional diesel flash points vary between 52 and 96 °C, which makes it safer than petrol and unsuitable for spark-ignition engines. Unlike petrol, the flash point of a diesel fuel has no relation to its performance in an engine nor to its auto ignition qualities.

Carbon dioxide formation

As a good approximation the chemical formula of diesel is C
_nH
_2n. Note that diesel is a mixture of different molecules. As carbon has a molar mass of 12 g/mol and hydrogen has a molar mass of about 1 g/mol, so the fraction by weight of carbon in EN 590 diesel fuel is roughly 12/14.

The reaction of diesel combustion is given by:

2C
_nH
_2n + 3nO
₂ ⇌ 2nCO
₂ + 2nH
₂O

Carbon dioxide has a molar mass of 44g/mol as it consists of 2 atoms of oxygen (16 g/mol) and 1 atom of carbon (12 g/mol). So 12 g of carbon yield 44 g of Carbon dioxide.

Diesel has a density of 0.838 kg per liter.

Putting everything together the mass of carbon dioxide that is produced by burning 1 liter of diesel fuel can be calculated as:

${\displaystyle 0.838kg/l\cdot {\frac {12}{14}}\cdot {\frac {44}{12}}=2.63kg/l}$

The figure obtained with this estimation is close to the values found in the literature.

For gasoline, with a density of 0.75 kg/l and a ratio of carbon to hydrogen atoms of about 6 to 14, the estimated value of carbon emission if 1 liter of gasoline is burnt gives:

${\displaystyle 0.75kg/l\cdot {{\frac {6\cdot 12}{6\cdot 12+14}}\cdot 1}\cdot {\frac {44}{12}}=2.3kg/l}$

Hazards

See also: diesel exhaust

Environment hazards of sulfur

In the past, diesel fuel contained higher quantities of sulfur. European emission standards and preferential taxation have forced oil refineries to dramatically reduce the level of sulfur in diesel fuels. In the European Union, the sulfur content has dramatically reduced during the last 20 years. Automotive diesel fuel is covered in the European Union by standard EN 590. In the 1990s specifications allowed a content of 2000 ppm max of sulfur, reduced to a limit of 350 ppm by the beginning of the 21st century with the introduction of Euro 3 specifications. The limit was lowered with the introduction of Euro 4 by 2006 to 50 ppm (ULSD, Ultra Low Sulfur Diesel). The standard for diesel fuel in force in Europe as of 2009 is the Euro 5, with a maximum content of 10 ppm.

Emission standard	At latest	Sulfur content	Cetane number
N/a	1 January 1994	max. 2000 ppm	min. 49
Euro 2	1 January 1996	max. 500 ppm	min. 49
Euro 3	1 January 2001	max. 350 ppm	min. 51
Euro 4	1 January 2006	max. 50 ppm	min. 51
Euro 5	1 January 2009	max. 10 ppm	min. 51

In the United States, more stringent emission standards have been adopted with the transition to ULSD starting in 2006, and becoming mandatory on June 1, 2010 (see also diesel exhaust).

Algae, microbes, and water contamination

Main article: microbial contamination of diesel fuel

There has been much discussion and misunderstanding of algae in diesel fuel. Algae need light to live and grow. As there is no sunlight in a closed fuel tank, no algae can survive, but some microbes can survive and feed on the diesel fuel.

These microbes form a colony that lives at the interface of fuel and water. They grow quite fast in warmer temperatures. They can even grow in cold weather when fuel tank heaters are installed. Parts of the colony can break off and clog the fuel lines and fuel filters.

Water in fuel can damage a fuel injection pump. Some diesel fuel filters also trap water. Water contamination in diesel fuel can lead to freezing while in the fuel tank. The freezing water that saturates the fuel will sometimes clog the fuel injector pump. Once the water inside the fuel tank has started to freeze, gelling is more likely to occur. When the fuel is gelled it is not effective until the temperature is raised and the fuel returns to a liquid state.

Road hazard

Diesel is less flammable than gasoline / petrol. However, because it evaporates slowly, any spills on a roadway can pose a slip hazard to vehicles. After the light fractions have evaporated, a greasy slick is left on the road which reduces tire grip and traction, and can cause vehicles to skid. The loss of traction is similar to that encountered on black ice, resulting in especially dangerous situations for two-wheeled vehicles, such as motorcycles and bicycles, in roundabouts.

Communication protocol

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Communication_protocol 

A communication protocol is a system of rules that allows two or more entities of a communications system to transmit information via any kind of variation of a physical quantity. The protocol defines the rules, syntax, semantics and synchronization of communication and possible error recovery methods. Protocols may be implemented by hardware, software, or a combination of both.

Communicating systems use well-defined formats for exchanging various messages. Each message has an exact meaning intended to elicit a response from a range of possible responses pre-determined for that particular situation. The specified behavior is typically independent of how it is to be implemented. Communication protocols have to be agreed upon by the parties involved. To reach an agreement, a protocol may be developed into a technical standard. A programming language describes the same for computations, so there is a close analogy between protocols and programming languages: protocols are to communication what programming languages are to computations. An alternate formulation states that protocols are to communication what algorithms are to computation.

Multiple protocols often describe different aspects of a single communication. A group of protocols designed to work together is known as a protocol suite; when implemented in software they are a protocol stack.

Internet communication protocols are published by the Internet Engineering Task Force (IETF). The IEEE (Institute of Electrical and Electronics Engineers) handles wired and wireless networking and the International Organization for Standardization (ISO) handles other types. The ITU-T handles telecommunications protocols and formats for the public switched telephone network (PSTN). As the PSTN and Internet converge, the standards are also being driven towards convergence.

Communicating systems

History

One of the first uses of the term protocol in a data-commutation context occurs in a memorandum entitled A Protocol for Use in the NPL Data Communications Network written by Roger Scantlebury and Keith Bartlett in April 1967.

On the ARPANET, the starting point for host-to-host communication in 1969 was the 1822 protocol, which defined the transmission of messages to an IMP. The Network Control Protocol (NCP) for the ARPANET was first implemented in 1970. The NCP interface allowed application software to connect across the ARPANET by implementing higher-level communication protocols, an early example of the protocol layering concept.

Networking research in the early 1970s by Robert E. Kahn and Vint Cerf led to the formulation of the Transmission Control Program (TCP). Its RFC 675 specification was written by Cerf with Yogen Dalal and Carl Sunshine in December 1974, still a monolithic design at this time.

The International Networking Working Group agreed a connectionless datagram standard which was presented to the CCIT in 1975 but was not adopted by the ITU or by the ARPANET. International research, particularly the work of Rémi Després, contributed to the development of the X.25 standard, based on virtual circuits by the ITU-T in 1976. Computer manufacturers developed proprietary protocols such as IBM's Systems Network Architecture (SNA), Digital Equipment Corporation's DECnet and Xerox Network Systems.

TCP software was redesigned as a modular protocol stack. Originally referred to as IP/TCP, it was installed on SATNET in 1982 and on the ARPANET in January 1983. The development of a complete protocol suite by 1989, as outlined in RFC 1122 and RFC 1123, laid the foundation for the growth of TCP/IP as a comprehensive protocol suite as the core component of the emerging Internet.

International work on a reference model for communication standards led to the OSI model, published in 1984. For a period in the late 1980s and early 1990s, engineers, organizations and nations became polarized over the issue of which standard, the OSI model or the Internet protocol suite, would result in the best and most robust computer networks.

Concept

The information exchanged between devices through a network or other media is governed by rules and conventions that can be set out in communication protocol specifications. The nature of communication, the actual data exchanged and any state-dependent behaviors, is defined by these specifications. In digital computing systems, the rules can be expressed by algorithms and data structures. Protocols are to communication what algorithms or programming languages are to computations.

Operating systems usually contain a set of cooperating processes that manipulate shared data to communicate with each other. This communication is governed by well-understood protocols, which can be embedded in the process code itself. In contrast, because there is no shared memory, communicating systems have to communicate with each other using a shared transmission medium. Transmission is not necessarily reliable, and individual systems may use different hardware or operating systems.

To implement a networking protocol, the protocol software modules are interfaced with a framework implemented on the machine's operating system. This framework implements the networking functionality of the operating system. When protocol algorithms are expressed in a portable programming language the protocol software may be made operating system independent. The best-known frameworks are the TCP/IP model and the OSI model.

At the time the Internet was developed, abstraction layering had proven to be a successful design approach for both compiler and operating system design and, given the similarities between programming languages and communication protocols, the originally monolithic networking programs were decomposed into cooperating protocols. This gave rise to the concept of layered protocols which nowadays forms the basis of protocol design.

Systems typically do not use a single protocol to handle a transmission. Instead they use a set of cooperating protocols, sometimes called a protocol suite. Some of the best known protocol suites are TCP/IP, IPX/SPX, X.25, AX.25 and AppleTalk.

The protocols can be arranged based on functionality in groups, for instance, there is a group of transport protocols. The functionalities are mapped onto the layers, each layer solving a distinct class of problems relating to, for instance: application-, transport-, internet- and network interface-functions. To transmit a message, a protocol has to be selected from each layer. The selection of the next protocol is accomplished by extending the message with a protocol selector for each layer.

Types

There are two types of communication protocols, based on their representation of the content being carried: text-based and binary.

Text-based

A text-based protocol or plain text protocol represents its content in human-readable format, often in plain text.

The immediate human readability stands in contrast to binary protocols which have inherent benefits for use in a computer environment (such as ease of mechanical parsing and improved bandwidth utilization).

Network applications have various methods of encapsulating data. One method very common with Internet protocols is a text oriented representation that transmits requests and responses as lines of ASCII text, terminated by a newline character (and usually a carriage return character). Examples of protocols that use plain, human-readable text for its commands are FTP (File Transfer Protocol), SMTP (Simple Mail Transfer Protocol), and the finger protocol.

Text-based protocols are typically optimized for human parsing and interpretation, and are therefore suitable whenever human inspection of protocol contents is required, such as during debugging and during early protocol development design phases.

To be clear, all digital communication is fundamentally binary. The “Text” based protocols mentioned here use only binary content, which is made “humanly readable” by a text editor (or other such software).

Binary

A binary protocol utilizes all values of a byte, as opposed to a text-based protocol which only uses values corresponding to human-readable characters in ASCII encoding. Binary protocols are intended to be read by a machine rather than a human being. Binary protocols have the advantage of terseness, which translates into speed of transmission and interpretation.

Binary have been used in the normative documents describing modern standards like EbXML, HTTP/2, HTTP/3 and EDOC. An interface in UML may also be considered a binary protocol.

Basic requirements

Getting the data across a network is only part of the problem for a protocol. The data received has to be evaluated in the context of the progress of the conversation, so a protocol must include rules describing the context. These kind of rules are said to express the syntax of the communication. Other rules determine whether the data is meaningful for the context in which the exchange takes place. These kind of rules are said to express the semantics of the communication.

Messages are sent and received on communicating systems to establish communication. Protocols should therefore specify rules governing the transmission. In general, much of the following should be addressed:

Data formats for data exchange

Digital message bitstrings are exchanged. The bitstrings are divided in fields and each field carries information relevant to the protocol. Conceptually the bitstring is divided into two parts called the header and the payload. The actual message is carried in the payload. The header area contains the fields with relevance to the operation of the protocol. Bitstrings longer than the maximum transmission unit (MTU) are divided in pieces of appropriate size.

Address formats for data exchange

Addresses are used to identify both the sender and the intended receiver(s). The addresses are carried in the header area of the bitstrings, allowing the receivers to determine whether the bitstrings are of interest and should be processed or should be ignored. A connection between a sender and a receiver can be identified using an address pair (sender address, receiver address). Usually, some address values have special meanings. An all-1s address could be taken to mean an addressing of all stations on the network, so sending to this address would result in a broadcast on the local network. The rules describing the meanings of the address value are collectively called an addressing scheme.

Address mapping

Sometimes protocols need to map addresses of one scheme on addresses of another scheme. For instance to translate a logical IP address specified by the application to an Ethernet MAC address. This is referred to as address mapping.

Routing

When systems are not directly connected, intermediary systems along the route to the intended receiver(s) need to forward messages on behalf of the sender. On the Internet, the networks are connected using routers. The interconnection of networks through routers is called internetworking.

Detection of transmission errors

Error detection is necessary on networks where data corruption is possible. In a common approach, a CRC of the data area is added to the end of packets, making it possible for the receiver to detect differences caused by corruption. The receiver rejects the packets on CRC differences and arranges somehow for retransmission.

Acknowledgements

Acknowledgement of correct reception of packets is required for connection-oriented communication. Acknowledgments are sent from receivers back to their respective senders.

Loss of information - timeouts and retries

Packets may be lost on the network or be delayed in transit. To cope with this, under some protocols, a sender may expect an acknowledgment of correct reception from the receiver within a certain amount of time. Thus, on timeouts, the sender may need to retransmit the information. In case of a permanently broken link, the retransmission has no effect so the number of retransmissions is limited. Exceeding the retry limit is considered an error.

Direction of information flow

Direction needs to be addressed if transmissions can only occur in one direction at a time as on half-duplex links or from one sender at a time as on a shared medium. This is known as media access control. Arrangements have to be made to accommodate the case of collision or contention where two parties respectively simultaneously transmit or wish to transmit.

Sequence control

If long bitstrings are divided into pieces and then sent on the network individually, the pieces may get lost or delayed or, on some types of networks, take different routes to their destination. As a result, pieces may arrive out of sequence. Retransmissions can result in duplicate pieces. By marking the pieces with sequence information at the sender, the receiver can determine what was lost or duplicated, ask for necessary retransmissions and reassemble the original message.

Flow control

Flow control is needed when the sender transmits faster than the receiver or intermediate network equipment can process the transmissions. Flow control can be implemented by messaging from receiver to sender.

Queueing

Communicating processes or state machines employ queues (or "buffers"), usually FIFO queues, to deal with the messages in the order sent, and may sometimes have multiple queues with different prioritization

Protocol design

Systems engineering principles have been applied to create a set of common network protocol design principles. The design of complex protocols often involves decomposition into simpler, cooperating protocols. Such a set of cooperating protocols is sometimes called a protocol family or a protocol suite, within a conceptual framework.

Communicating systems operate concurrently. An important aspect of concurrent programming is the synchronization of software for receiving and transmitting messages of communication in proper sequencing. Concurrent programming has traditionally been a topic in operating systems theory texts. Formal verification seems indispensable because concurrent programs are notorious for the hidden and sophisticated bugs they contain. A mathematical approach to the study of concurrency and communication is referred to as communicating sequential processes (CSP). Concurrency can also be modeled using finite state machines, such as Mealy and Moore machines. Mealy and Moore machines are in use as design tools in digital electronics systems encountered in the form of hardware used in telecommunication or electronic devices in general.

The literature presents numerous analogies between computer communication and programming. In analogy, a transfer mechanism of a protocol is comparable to a central processing unit (CPU). The framework introduces rules that allow the programmer to design cooperating protocols independently of one another.

Layering

Figure 2. The TCP/IP model or Internet layering scheme and its relation to some common protocols.

In modern protocol design, protocols are layered to form a protocol stack. Layering is a design principle that divides the protocol design task into smaller steps, each of which accomplishes a specific part, interacting with the other parts of the protocol only in a small number of well-defined ways. Layering allows the parts of a protocol to be designed and tested without a combinatorial explosion of cases, keeping each design relatively simple.

The communication protocols in use on the Internet are designed to function in diverse and complex settings. Internet protocols are designed for simplicity and modularity and fit into a coarse hierarchy of functional layers defined in the Internet Protocol Suite. The first two cooperating protocols, the Transmission Control Protocol (TCP) and the Internet Protocol (IP) resulted from the decomposition of the original Transmission Control Program, a monolithic communication protocol, into this layered communication suite.

The OSI model was developed internationally based on experience with networks that predated the internet as a reference model for general communication with much stricter rules of protocol interaction and rigorous layering.

Typically, application software is built upon a robust data transport layer. Underlying this transport layer is a datagram delivery and routing mechanism that is typically connectionless in the Internet. Packet relaying across networks happens over another layer that involves only network link technologies, which are often specific to certain physical layer technologies, such as Ethernet. Layering provides opportunities to exchange technologies when needed, for example, protocols are often stacked in a tunneling arrangement to accommodate the connection of dissimilar networks. For example, IP may be tunneled across an Asynchronous Transfer Mode (ATM) network.

Protocol layering

Figure 3. Message flows using a protocol suite. Black loops show the actual messaging loops, red loops are the effective communication between layers enabled by the lower layers.

Protocol layering forms the basis of protocol design. It allows the decomposition of single, complex protocols into simpler, cooperating protocols. The protocol layers each solve a distinct class of communication problems. Together, the layers make up a layering scheme or model.

Computations deal with algorithms and data; Communication involves protocols and messages; So the analog of a data flow diagram is some kind of message flow diagram. To visualize protocol layering and protocol suites, a diagram of the message flows in and between two systems, A and B, is shown in figure 3. The systems, A and B, both make use of the same protocol suite. The vertical flows (and protocols) are in-system and the horizontal message flows (and protocols) are between systems. The message flows are governed by rules, and data formats specified by protocols. The blue lines mark the boundaries of the (horizontal) protocol layers.

Software layering

Figure 5: Protocol and software layering. The software modules implementing the protocols are represented by cubes. The information flow between the modules is represented by arrows. The (top two horizontal) red arrows are virtual. The blue lines mark the layer boundaries.

The software supporting protocols has a layered organization and its relationship with protocol layering is shown in figure 5.

To send a message on system A, the top-layer software module interacts with the module directly below it and hands over the message to be encapsulated. The lower module fills in the header data in accordance with the protocol it implements and interacts with the bottom module which sends the message over the communications channel to the bottom module of system B. On the receiving system B the reverse happens, so ultimately the message gets delivered in its original form to the top module of system B.

Program translation is divided into subproblems. As a result, the translation software is layered as well, allowing the software layers to be designed independently. The same approach can be seen in the TCP/IP layering.

The modules below the application layer are generally considered part of the operating system. Passing data between these modules is much less expensive than passing data between an application program and the transport layer. The boundary between the application layer and the transport layer is called the operating system boundary.

Strict layering

Strictly adhering to a layered model, a practice known as strict layering, is not always the best approach to networking. Strict layering can have a negative impact on the performance of an implementation.

While the use of protocol layering is today ubiquitous across the field of computer networking, it has been historically criticized by many researchers as abstracting the protocol stack in this way may cause a higher layer to duplicate the functionality of a lower layer, a prime example being error recovery on both a per-link basis and an end-to-end basis.

Design patterns

Commonly recurring problems in the design and implementation of communication protocols can be addressed by software design patterns.

Formal specification

Popular formal methods of describing communication syntax are Abstract Syntax Notation One (an ISO standard) and augmented Backus–Naur form (an IETF standard).

Finite-state machine models are used to formally describe the possible interactions of the protocol and communicating finite-state machines.

Protocol development

For communication to occur, protocols have to be selected. The rules can be expressed by algorithms and data structures. Hardware and operating system independence is enhanced by expressing the algorithms in a portable programming language. Source independence of the specification provides wider interoperability.

Protocol standards are commonly created by obtaining the approval or support of a standards organization, which initiates the standardization process. The members of the standards organization agree to adhere to the work result on a voluntary basis. Often the members are in control of large market-shares relevant to the protocol and in many cases, standards are enforced by law or the government because they are thought to serve an important public interest, so getting approval can be very important for the protocol.

The need for protocol standards

The need for protocol standards can be shown by looking at what happened to the bi-sync protocol (BSC) invented by IBM. BSC is an early link-level protocol used to connect two separate nodes. It was originally not intended to be used in a multinode network, but doing so revealed several deficiencies of the protocol. In the absence of standardization, manufacturers and organizations felt free to enhance the protocol, creating incompatible versions on their networks. In some cases, this was deliberately done to discourage users from using equipment from other manufacturers. There are more than 50 variants of the original bi-sync protocol. One can assume, that a standard would have prevented at least some of this from happening.

In some cases, protocols gain market dominance without going through a standardization process. Such protocols are referred to as de facto standards. De facto standards are common in emerging markets, niche markets, or markets that are monopolized (or oligopolized). They can hold a market in a very negative grip, especially when used to scare away competition. From a historical perspective, standardization should be seen as a measure to counteract the ill-effects of de facto standards. Positive exceptions exist; a de facto standard operating system like Linux does not have this negative grip on its market, because the sources are published and maintained in an open way, thus inviting competition.

Standards organizations

Some of the standards organizations of relevance for communication protocols are the International Organization for Standardization (ISO), the International Telecommunication Union (ITU), the Institute of Electrical and Electronics Engineers (IEEE), and the Internet Engineering Task Force (IETF). The IETF maintains the protocols in use on the Internet. The IEEE controls many software and hardware protocols in the electronics industry for commercial and consumer devices. The ITU is an umbrella organization of telecommunication engineers designing the public switched telephone network (PSTN), as well as many radio communication systems. For marine electronics the NMEA standards are used. The World Wide Web Consortium (W3C) produces protocols and standards for Web technologies.

International standards organizations are supposed to be more impartial than local organizations with a national or commercial self-interest to consider. Standards organizations also do research and development for standards of the future. In practice, the standards organizations mentioned, cooperate closely with each other.

The standardization process

In the ISO, the standardization process starts off with the commissioning of a sub-committee workgroup. The workgroup issues working drafts and discussion documents to interested parties (including other standards bodies) in order to provoke discussion and comments. This will generate a lot of questions, much discussion and usually some disagreement. These comments are taken into account and a draft proposal is produced by the working group. After feedback, modification, and compromise the proposal reaches the status of a draft international standard, and ultimately an international standard. International standards are reissued periodically to handle the deficiencies and reflect changing views on the subject.

OSI standardization

A lesson learned from ARPANET, the predecessor of the Internet, was that protocols need a framework to operate. It is therefore important to develop a general-purpose, future-proof framework suitable for structured protocols (such as layered protocols) and their standardization. This would prevent protocol standards with overlapping functionality and would allow clear definition of the responsibilities of a protocol at the different levels (layers). This gave rise to the Open Systems Interconnection model (OSI model), which is used as a framework for the design of standard protocols and services conforming to the various layer specifications.

In the OSI model, communicating systems are assumed to be connected by an underlying physical medium providing a basic transmission mechanism. The layers above it are numbered. Each layer provides service to the layer above it using the services of the layer immediately below it. The top layer provides services to the application process. The layers communicate with each other by means of an interface, called a service access point. Corresponding layers at each system are called peer entities. To communicate, two peer entities at a given layer use a protocol specific to that layer which is implemented by using services of the layer below. For each layer, there are two types of standards: protocol standards defining how peer entities at a given layer communicate, and service standards defining how a given layer communicates with the layer above it.

In the OSI model, the layers and their functionality are (from highest to lowest layer):

• The Application layer may provide the following services to the application processes: identification of the intended communication partners, establishment of the necessary authority to communicate, determination of availability and authentication of the partners, agreement on privacy mechanisms for the communication, agreement on responsibility for error recovery and procedures for ensuring data integrity, synchronization between cooperating application processes, identification of any constraints on syntax (e.g. character sets and data structures), determination of cost and acceptable quality of service, selection of the dialogue discipline, including required logon and logoff procedures.
• The presentation layer may provide the following services to the application layer: a request for the establishment of a session, data transfer, negotiation of the syntax to be used between the application layers, any necessary syntax transformations, formatting and special purpose transformations (e.g. data compression and data encryption).
• The session layer may provide the following services to the presentation layer: establishment and release of session connections, normal and expedited data exchange, a quarantine service which allows the sending presentation entity to instruct the receiving session entity not to release data to its presentation entity without permission, interaction management so presentation entities can control whose turn it is to perform certain control functions, resynchronization of a session connection, reporting of unrecoverable exceptions to the presentation entity.
• The transport layer provides reliable and transparent data transfer in a cost-effective way as required by the selected quality of service. It may support the multiplexing of several transport connections on to one network connection or split one transport connection into several network connections.
• The network layer does the setup, maintenance and release of network paths between transport peer entities. When relays are needed, routing and relay functions are provided by this layer. The quality of service is negotiated between network and transport entities at the time the connection is set up. This layer is also responsible for network congestion control.
• The data link layer does the setup, maintenance and release of data link connections. Errors occurring in the physical layer are detected and may be corrected. Errors are reported to the network layer. The exchange of data link units (including flow control) is defined by this layer.
• The physical layer describes details like the electrical characteristics of the physical connection, the transmission techniques used, and the setup, maintenance and clearing of physical connections.

In contrast to the TCP/IP layering scheme, which assumes a connectionless network, RM/OSI assumed a connection-oriented network. Connection-oriented networks are more suitable for wide area networks and connectionless networks are more suitable for local area networks. Connection-oriented communication requires some form of session and (virtual) circuits, hence the (in the TCP/IP model lacking) session layer. The constituent members of ISO were mostly concerned with wide area networks, so the development of RM/OSI concentrated on connection-oriented networks and connectionless networks were first mentioned in an addendum to RM/OSI and later incorporated into an update to RM/OSI.

At the time, the IETF had to cope with this and the fact that the Internet needed protocols that simply were not there. As a result, the IETF developed its own standardization process based on "rough consensus and running code". The standardization process is described by RFC 2026.

Nowadays, the IETF has become a standards organization for the protocols in use on the Internet. RM/OSI has extended its model to include connectionless services and because of this, both TCP and IP could be developed into international standards.

Taxonomies

Classification schemes for protocols usually focus on the domain of use and function. As an example of domain of use, connection-oriented protocols and connectionless protocols are used on connection-oriented networks and connectionless networks respectively. An example of function is a tunneling protocol, which is used to encapsulate packets in a high-level protocol so that the packets can be passed across a transport system using the high-level protocol.

A layering scheme combines both function and domain of use. The dominant layering schemes are the ones developed by the IETF and by ISO. Despite the fact that the underlying assumptions of the layering schemes are different enough to warrant distinguishing the two, it is a common practice to compare the two by relating common protocols to the layers of the two schemes. The layering scheme from the IETF is called Internet layering or TCP/IP layering. The layering scheme from ISO is called the OSI model or ISO layering.

In networking equipment configuration, a term-of-art distinction is often drawn: The term protocol strictly refers to the transport layer, and the term service refers to protocols utilizing a protocol for transport. In the common case of TCP and UDP, services are distinguished by port numbers. Conformance to these port numbers is voluntary, so in content inspection systems the term service strictly refers to port numbers, and the term application is often used to refer to protocols identified through inspection signatures.