|
12th Annual Carolina Swine Nutrition Conference, November 13th, 1996 NIR technology: Can we measure amino acid digestibility and energy values? Theo van Kempen, Rhône-Poulenc Animal Nutrition, Antony, France Bottom line: The
NIRS may be used as a tool to predict the digestible amino acid and likely the
energy content of feedstuffs. Such
information is useful from a quality control perspective to find outliers in
incoming raw material and to rank batches of feedstuffs based on nutritionally
relevant parameters; for a nutritionist, such information may be used, in
combination with laboratory data, to reformulate diets.
Introduction The feed industry requires
rapid and accurate information concerning the nutritive value of feedstuffs. On
one hand, such information is needed to negotiate the proper price for a
feedstuff, and on the other hand, to correctly include this feedstuff in a
complete diet such that the animal’s nutrition requirements are met with the
lowest feed cost. Preferably, such information is obtained rapidly so that a
routine evaluation of a feedstuff is possible. Currently, many feedmills
use NIRS technology to predict the protein, moisture, fat, and ash contents of
feedstuffs to obtain the needed information on a feedstuff. However, these
parameters are of limited value since they only correlate to a limited extend
with the true nutritional value of a feedstuff. Better parameters to evaluate
are the amino acid availability, and the net energy of a feedstuffs. The objective of this paper
is to review the basics of NIRS and the potential of this technique as a tool
to routinely evaluate feedstuffs for nutritionally relevant parameters. Part I:
Understanding NIRS Color To understand the basic
principle of NIRS, it is necessary to understand the basis of color. White
light is composed of all the colors in the rainbow. When this white light falls
onto an object, certain colors within this spectrum are absorbed by the object,
while the rest is reflected (or transmitted). The light reflected can be
observed by the eye, and its composite remaining color is interpreted by the
eye as the color of the object. As an example, when sunlight
falls onto a clover leaf, both the red and the blue light are absorbed to aid
in photosynthesis. The other colors are reflected, and are interpreted by the
eye as green. Physics
behind light absorption The physical basis of light
absorption is linked to the nature of molecular bonds and to the nature of
light. Light consists of photons traveling all at a fixed speed. While
traveling, these photons exhibit a ‘wave’ motion around the path of travel.
This wave motion is specific for each color of light, and is often expressed as
the frequency (the number of waves traveled per second), or as the wavelength
(the distance traveled while completing 1 ‘wave’). Molecular bonds are the links
between atoms within a molecule. These bonds are not static connections, since
the atoms they connect vibrate relative to each other. This vibration causes
the molecular bond to stretch and compress, resulting in a wave motion of the
two atoms relative to each other with a frequency specific to the atoms
involved. When light hits such a
vibrating bond, two situations may arise: 1) When the
frequency (‘color’) of this light does not match the frequency of the vibration
of the molecular bond, then this light is reflected 2) When the
frequency (‘color’) of this light matches the frequency of the vibration of the
molecular bond, then this light is absorbed, and its energy is used to
intensify the vibration (the objects becomes warmer). Light
absorption by feedstuffs Feedstuffs are made up of
mainly organic matter. The molecular bonds which are most common in a feedstuff
are thus bonds between hydrogen, carbon, oxygen, sulfur, phosphorus, and
nitrogen. The frequency of the vibration between these molecules is such that
these bonds generally absorb light in the near-infrared region, or the region
which extends just beyond red in the rainbow (not visible for humans). NIRS uses the principle that
molecular bonds absorb specific frequencies of light to obtain information
about the number and type of organic bonds present in a feedstuff; in other
words, by ‘looking’ at the feedstuff’s color in the near-infrared region. NIRS
Equipment The principle of a NIRS
machine is that a feedstuff is illuminated with light of a specific and known
frequency (or wavelength) in the near-infrared region. The absorption of light
by the feedstuff is then measured as the difference between the amount of light
emitted by the NIRS machine and the amount of light reflected by the sample.
The machine layout is explained in Figure 1. ![]()
Fig. 1. Schematical layout of NIRS machine. Light is focused (1)
onto a concave mirror (2), which separates the light into its composing
wavelengths (3). One wave-length is selected (4), and falls onto the feedstuff
(5). The amount of this light which is reflected by the feedstuff is measured
(6) to obtain the absorption corresponding to one wavelength. By changing the
orientation of the mirror (2), different wavelengths of light are selected to
obtain absorption measures for all wavelengths of interest. The graph depicting
the wavelength tested versus the absorption is called the spectrum, and an
example spectrum for coconut meal is presented in the top right corner (7). When it is known at which
frequency specific molecular bonds vibrate then it is possible to deduce from a
spectrum the composition of the molecular bonds present in a sample evaluated.
Using NIRS to identify simple molecules works indeed that easily. The NIRS spectrum presented
in Fig. 2 is that obtained from DL-methionine. The peaks present in this
spectrum are related to the C-H bonds, and tables exist to relate the
wavelength at which these peaks were found to the chemical composition of the
sample. Spectral information can thus be used to determine the identity of
samples. ![]()
Fig. 2. NIRS spectrum of methionine. The peaks correspond to C-H bonds in
the sample. Interpretation
of feedstuff spectra Unfortunately, for
feedstuffs the picture is more complicated. A large number of molecular bonds
are present in a feedstuff, and at any given wavelength, many bonds will absorb
light. The resulting spectrum is thus a ‘composite’, where small peaks caused
by a specific molecular bond are non-distinguishable (see Fig. 3). This lack of
distinguishable peaks means that it is impossible to deduce the composition of
a sample from spectral information using tables which indicate which wavelength
corresponds with which type of molecular bond. ![]()
Fig. 3. NIRS spectrum of soybean meal. No specific peaks are
present which allow for a direct quantification of the sample’s composition. Instead, to predict the
composition of feedstuffs it is necessary to make calibrations. A calibration
is simply NIRS spectral information correlated with composition values
(obtained by, e.g., wet chemistry). This concept is most easily explained by a
fictitious example (see Fig. 4). Let us imagine the spectra obtained for 3
soybean samples which are identical except for their protein content. The
protein content of these samples was either 44%, 46%, or 48%. In this example,
the spectra from these samples deviates only in the region of 1900-2000
nanometer (nm; the wavelength of the light), and this difference must be
related to the differences in protein content of the sample, since the samples
were otherwise identical. Plotting the absorption at
1950 nm for each spectrum against the protein level shows that a linear
relationship exists between the absorption of light at 1950 nm and the protein
level. It is easy to image that such a correlation could be used to predict the
protein content of a fourth soybean sample. In reality, spectra are much
more complicated. Protein does not absorb at only one region of the spectra (as
seen for methionine in Fig. 2). Additionally, when protein levels increase in a
sample, other compounds, e.g., carbohydrates, must decrease, and thus other
regions of the spectrum must change. The spectra of the three samples should
thus be different in several regions, and it is not so easy to identify a
region of the spectrum in which absorption correlates with the protein content
of the samples. Nevertheless, this approach is basically the outlined procedure
which is used to determine the correlations between spectral information and
chemical composition, which are then subsequently used as calibrations to
predict unknown samples. ![]()
Fig. 4 Fictitious example to illustrate the principle of
calibrating: Spectral information is correlated with analytical information
(see text for details). Such calibrations have been
successfully applied in the feed industry to predict parameters such as
protein, moisture, NDF, ADF, etc. The main reason for this success is the
rapidity of the measurement: values can be obtained within 5 minutes of
receiving a sample. Calibration
validation In the process of
calculating correlations, the NIRS software has no notion of the functionality
of the correlations it determines. When a spectral peak caused by the presence
of carbohydrates happens to correlate (negatively) with the protein level in the
reference samples, then a calibration might well be developed based on this
carbohydrate peak. Such a calibration is not
very useful. Although a relationship exists between carbohydrates and protein
in the reference population, this relationship does not hold in the general
population. It is thus essential to prevent the use of such calibrations in
real-life applications. In order to improve the
chance of developing valid calibrations, the reference database should contain
the maximum amount of variation. In practical terms, at least 100 different
samples should be used. These samples must be representative of the population
to be predicted in the future, and should thus be obtained from different
harvest years, the geographical regions from which this feedstuff is obtained,
and cover all the genotypes of interest. The actual functionality of
a calibration should be tested in a process called validation. Ideally, a
number of samples covering the range of samples which can be encountered in
real life are used. All these samples are predicted using the calibration to be
tested, and analyzed with the reference technique to obtain the true values.
The difference between the predicted and true value is used to calculate the
root mean square error of prediction (RMSEP), which is a measure for the
accuracy of the calibration. However, this exercise is
tedious and costly, and because of this, NIRS software developers have included
ingenious validation procedures in their software. It never hurts, though, to
check the validity of a calibration for yourself. Part II:
NIRS to predict the energy content of feedstuffs Several research groups have
developed NIRS calibrations for energy values, however, the majority of this
work has been focused on ruminants. Energy predictions for forages are common
and are utilized commercially, which suggests that energy can be predicted with
an adequate precision (DeBoever et al. 1993). However, the energy content of
ruminant feeds is often determined by proximate analyses in combination with
some factor of rumen or in vitro digestibility, rather than by in vivo
assays. For poultry, apparent
metabolizable energy (AME) predictions were obtained with a prediction error of
80 kcal/kg feed by Valdez and Leeson (1992). However, in the sample population
tested by Valdez the AME value of the feedstuffs was linearly dependent on the
fat content of the test diet. The utility of NIRS to predict the fat content
has been well documented, but unfortunately this does also imply that the
calibration generated is not necessarily effective in predicting AME in cases
where the fat content does not correlate with the AME content of a diet. Flinn
et al. (1994) obtained a prediction error for AME ranging from 90 kcal/kg DM
for cereals, to 275 kcal/kg for mixed feeds. This prediction error, however,
appears close to the normal variability observed. For pigs, digestible energy
calibrations were developed by Van der Meer et al. (1988). This paper, though,
only lists the standard error of the calibration (a measure of how well the
calibration describes the population used to make the calibration), which is of
little value for the true prediction error of a calibration. The success of
NIRS in the ruminant area suggests, nevertheless, that the NIRS is capable of
predicting energy values for feeds or feedstuffs for swine. In addition, the
theoretical basis of NIRS suggests that energy predictions should be
achievable. Until a serious attempt to do so is published, though, it remains
to be seen if it is feasible in real life. Part III:
NIRS to predict the digestible amino acid content of feedstuffs Calibration
development Since 1981, Rhône-Poulenc
Animal Nutrition has tested feedstuffs in poultry for ileal amino acid
digestibility according to the procedure of Green et. al (1986). Most samples
used in these tests have been retained in cold storage. Currently, a large
database exists containing samples from virtually every category of feedstuffs
that would be of interest to the feedstuff industry and derived from many
countries around the world. The samples contained in
this database were used for the development of NIRS calibrations. Course
samples, such as grains, were ground using a Retch grinder (model ZM1000, Haan,
Germany) with a 1 mm screen, while the other samples were scanned as-is. Prior
to scanning, samples were acclimated to the NIRS room temperature of 23°C.
Scans were made using a NIRSystems model 6500 NIRS (Perstorp Analytical,
Höganäs, Sweden; a complete setup costs in the order of $75000) equipped with a
natural product cell. These scans were subsequently correlated to the ileal
digestibility data obtained in vivo. The validation procedure
used to test calibrations is as follows: one sample was deleted from the
database, after which a calibration was developed using all other samples in
the database. This calibration was subsequently used to predict the deleted
sample. This procedure was repeated for all the samples in the database, and
the difference between the true and predicted values was used to calculate the
prediction error (RMSEP or Root mean square error of prediction). Calibrations were developed
and tested for either specific feedstuffs (e.g., only meat & bone meal),
feedstuff categories (e.g., all animal protein products, including e.g., fish
meal, meat & bone meal, etc.), or a global database (containing all
feedstuffs). In principle, a calibration based on only one feedstuff yields the
best results. However, this requires that sufficient samples are present in the
database to cover all possible variation. Including samples from a different
category increases the variation, and may improve the prediction accuracy of
the calibration. Table I. Root mean square errors of prediction (RMSEP; g/100g) and
derived r²rmsep values for meat & bone meal within the global
(based on all feedstuffs), product group (based on feedstuffs of animal
origin), product calibration (based on meat & bone meal samples). (The
three-letter amino acid code followed by a ‘P’ indicates ‘poultry digestible’;
nd=not determined)
The validation results for
meat & bone meal samples for the three different database types tested are
provided in Table I. The RMSEP (expected prediction error) and r² values
(variation explained by the calibration) as presented in this table were equal
or better (lower for RMSEP, higher for r²) when using the global calibration in
comparison with the animal meal or meat & bone meal calibration. For the
other animal meals (fish meal and poultry byproducts), similar results were
obtained (results not shown). Based on this database comparison test, it can
thus be concluded that for animal meal samples, the global calibration outperformed the other calibrations. For products with lower
protein levels, such as cereals and oilseed meals, product and product group
calibrations performed better than the global calibration (data not
shown). These data illustrate that
it is important to consider which database is used to obtain a calibration.
Since the global database performed best for the animal meal samples, a global
calibration was subsequently developed for all total and essential amino acids.
The calibration and validation statistics for a part of this calibration are
shown in Table II. Table II. Population statistics and performance (g/100g) for a
part of the calibration developed using the ‘global’ database. (The
three-letter amino acid code followed by a ‘P’ indicates ‘poultry digestible’.)
In reality, however,
calibration and validation statistics should only be considered within product
categories. The explanation for this is that it is easy to explain variation
when categories exist within a database. As an example; if a calibration was
based on 10 maize and 10 soybean meal samples, then predicting 0.23% total
lysine for the maize and 2.7% total lysine for the soybean meal would likely
explain 90% of the variation of the population containing both the maize and
soybean samples, but none of the variation within the maize population, nor
within the soybean population. Such a prediction would be just as good as a
bookvalue. Table III. Calibration performance (g/100g) of the global
calibration as applied to meat & bone meal and fish meal. (The three-letter
amino acid code followed by a ‘P’ indicates ‘poultry digestible’.)
For this reason, calibration and validation
statistics within certain animal meals are provided in Table III, and based on
this table one can really decide what the value of a calibration is. This table
shows that for the amino acids listed, 60-90% of the variation within the
populations was explained by the global calibration. Advantages
of a global calibration The global calibration
performed most accurately for high-protein feedstuffs. Although it works for
other feedstuffs, the prediction error makes it unattractive for use for
feedstuffs with low levels of amino acids. Nevertheless, such
predictions may still be interesting when no or limited data are available for
a certain product. For example, samples of coconut meal, mung, and kapok beans
from Asia and an extruded soybean-rapeseed mix product from Europe were
received at the Rhône-Poulenc Laboratory for in vivo testing. For all
four samples, the global calibration predicted the composition with an accuracy
sufficient to get an idea of the nutritional value of the product. Another application might be
to use the global calibration for mixed feeds. Although no official tests have
been carried out with mixed feeds, it appears that the current calibrations
predict the total and digestible amino acid profiles of mixed feedstuffs with a
reasonable accuracy. However, synthetic amino acids are not detected as amino
acids (they are included in the protein estimate) since they were not included
in the calibration samples. NIRS
in comparison to other techniques To fully understand the
value of the current global calibration it is important to look at alternative
techniques for obtaining total and digestible amino acid values for feedstuffs.
A multitude of techniques exist, but most of them have either one or both of
the following disadvantages: they take too much time, or they are too
expensive. Techniques which are expensive and/or cumbersome can not be applied
as a routine evaluation system, and to compare the NIRS to such a technique is
not very meaningful. Instead, NIRS is to be used
in situations where a answer is needed quickly, and in such situations a
compromise with respect to accuracy is often unavoidable. As an example,
currently many feedmills use book values (often personalized) in their
formulations. Such a bookvalue, although accurate for a population mean, does
not explain any of the variation within a population. The prediction error
(RMSEP) in this case is thus equal to the standard deviation of the population,
and the explained variation (r²) is 0. A second alternative would
be to measure the protein content of a batch of a feedstuff, and to use
regression to predict the digestible amino acid content. The accuracies of such
regressions vary. For maize and wheat, our calculations indicate that they are
useless (r²msep =0), while in meat & bone meal they explain a
reasonable percentage of variation (r²rmsep=0.59). Other products
evaluated fall somewhere in between. This means that even in the best
regression prediction (meat & bone meal), regression yields results which
are still inferior to the NIRS predictions. So, as a rapid and practical
technique to predict total and digestible amino acid contents of feedstuffs the
NIRS scores quite well in comparison to some of the available alternatives
(Table IV). Table IV: Comparison (RMSEP in g/100g and derived r²rmsep
values) of NIRS digestible amino acid prediction accuracy with the accuracy of
other rapid techniques to predict the digestible amino acid content of animal
meal samples. The estimated in vivo accuracy (in g/100g) is provided
for comparison only, since this technique may not be considered as ‘rapid’.
Comparison
with in
vivo estimations When evaluating a
calibration, it is of importance to keep the quality of the reference data in
mind. Since the calibration was calculated based on correlations between
spectral and the (error containing) reference data. The errors of the digestible
amino acid assay are first, the amino acid analysis of the feedstuff, and
second, the error associated with the in vivo part of the digestibility assay.
Amino acids were analyzed by HPLC, and the error of this technique is generally
considered to ±5%. Subsequently, digestibility
was evaluated in vivo, and due to animal variation and the analysis of
ileal juice for amino acids this assay generally adds another 2% to the overall
error. The total error of the digestible amino acid assay may thus be estimated
at 7%, or for meat & bone meal, 0.20% of digestible lysine. The prediction error
obtained by NIRS ranged, for the data presented, from similar to up to 2 times
worse than the in vivo assay error (see Table IV). This is in the range
which is deemed acceptable for NIRS techniques. It should thus be concluded
that these first-generation calibrations are acceptable. Poultry as
a model for swine digestibility The calibration results
discussed were for poultry digestible lysine. Little work has been performed on
predicting swine ileal digestibility, the reason being that large databases of
feedstuff samples with accompanying swine ileal digestibility data are not
available. Work within our laboratory based on almost 100 feed samples tested
in both pigs and poultry, however, suggests that a relationship exists between
poultry digestible and swine digestible amino acid contents of feedstuffs. For
lysine, this relationship, which explained 99.7% of the variation, is depicted
in Fig. 6; for methionine, 99.7% of the variation was explained as well, and
the prediction error was 0.02% digestible methionine. For other amino acids,
relationships were obtained which explained at least 98% of the variability. Given the almost perfect
relationship between swine and poultry digestible amino acid contents of
feedstuffs, calibrations based on swine ileal digestible data should be able to
be generated. Alternatively, poultry predictions could be made, and these could
subsequently be converted to swine predictions. The success of such a
procedure, however, depends on the willingness of the industry to accept the
correlations between poultry and swine digestibility. ![]()
Fig. 6. Correlation between poultry digestible lysine and swine
digestible lysine based on 85 samples tested in both species. Poultry values
explain 99.7% of the variation in swine values, and the error introduced when
extrapolating swine digestibility values based on Poultry data is 0.08% (absolute
prediction error for lysine). Practical
use of NIRS calibrations In a commercial feedmill, it
is practically and technically not possible to adjust ration formulations based
on the nutritional profile of each batch of raw material received.
Additionally, no technique exists which can determine the digestible amino acid
profile of each batch of feedstuff with sufficient accuracy and within the cost
and time constraints of a feedmill. NIRS, however, would fit within the time
and cost constraints, and approaches the accuracy limit. From our perspective, the
presented NIRS calibrations are most interesting for monitoring the nutritional
profile of a feedstuff upon reception. This would allow the feedmill to detect
outliers which require additional analyses and which may require a re-evaluation
of a supplier or the price being paid for the product. Over time (when several
subsequent batches of a feedstuff have been predicted), in conjunction with
laboratory analyses, the NIRS data may be used to adjust formulations. Such a procedure also helps
to evaluate NIRS. When laboratory analyses confirm the NIRS results, the user
builds confidence in the NIRS predictions. Over time, the user will increase
his reliance on the NIRS technique, resulting in less samples being selected
for laboratory analysis. In the situation where a
feedmill has multiple bins for one type of a highly variable feedstuff in order
to classify incoming material, NIRS predictions may also be used to determine
in which quality category a certain batch belongs. The value of NIRS for
feedstuff categorization is illustrated in Figure 7, which is based on the
samples in the RPAN database. Meat & Bone meal was categorized based on its
digestible lysine content in the classes 1-2%, 2-3%, and 3-4% digestible
lysine. In the test, 30 of the 41 samples were categorized correctly by the
NIRS, 6 samples were ‘borderline’ (categorization error acceptable since it
falls within the limit of the reference technique), and 5 samples were categorized incorrectly (statistically,
74% should be categorized correctly when 80% of the variation in a population
is explained; Shenk & Westerhouse, 1993). ![]()
Fig. 7. NIRS as a bin selection tool for feedstuffs for which
different quality grades are distinguished. NIRS correctly classified, based on
the predicted digestible lysine content, the meat & bone meal samples
within the large filled-in squares. Within the small open squares, errors are
acceptable (within the accuracy limits of the reference method). Samples
outside the squares were classified incorrectly. Work in
progress As indicated above, our
initial work to calibrate the NIRS for digestible amino acids was successful.
The prediction errors obtained, although not perfect, were low enough so that
the calibrations become attractive for practical use in evaluating high protein
animal meals. Other calibrations (based on
product databases) have been developed for maize and soybean meals, and the
validation results of these calibrations suggest that it is possible to calibrate
the NIRS for these products as well. Digestibility trials to finalize these
calibrations are already underway. Selected
references DeBoever, J.L., Cottyn, B.G., Vanacker, J.M., and
Boucqué, Ch.V., 1993. Recent developments in the use of NIRS for evaluating
compound feeds and raw materials for ruminants. Paper 1; Proceedings Conference
NIR Spectroscopy - Developments in Agriculture and Food. Birmingham, UK. ADAS
Drayton Research Centre FEU. Flinn, P.C., Windham, W.R., Johnson, R.J., 1994.
Current status of near infrared (NIR) spectroscopy in Australia for predicting
metabolisable energy of poultry feeds.
Proceedings, 9th European Poultry Conference. Glasgow, UK. pp.106-109. Green, S., 1986. Digestibility of amino acids in
practical feed formulation. Poultry International, October: 140-143. Green, S., S. Bertrand, M. Duron, and Maillard, R.,
1987. Digestibilities of amino
acids in maize, wheat, and barley meals, determined with intact and
caecectomized cockerels. Br. Poultry Sci., 28: 631-641. van der Meer, J.M., Vedder, H., and Wever, G., 1988.
NIRS for the prediction of organic matter and energy digestibility in pig
feeds. In
vitro newsletter No. 4: 25-26. Murray, I., and Hall, A.P., 1983. Animal feed
evaluation by use of Near Infrared Reflectance (NIR) Spectrocomputer. Anal.
Proc. 20: 75-70. Low, A.G., 1990. In: J. Wiseman and D.J.A. Cole
(Editors), Protein evaluation in pigs and poultry in feedstuff evaluation.
Butterworths, London, pp. 91-114. Osborne, B.G., Fearn, T., and Hindle, P.H., 1993.
Practical NIR spectroscopy: with applications in food and beverage analysis
(2nd edition). Longman Singapore Publishers, Singapore. Rostagno, H.S., and Pupa, J.M.R., 1995. Diet
formulation for broilers based on total versus digestible amino acids. J. Appl.
Poultry Res. 4: 293-299. Rhône-Poulenc, 1993. Rhodimet nutrition guide, 2nd
edition. Rhône-Poulenc Animal Nutrition, Antony, France. Shenk, J.S., and Westerhaus, M.O., 1993. Monograph.
Analysis of agriculture and food products by near infrared reflectance spectroscopy.
Infrasoft International, Port Matilda, PA, USA. Valdez, E.V., and Leeson, S., 1992. Near infrared
reflectance analysis as a method to measure metabolizable energy in complete
poultry feeds. Poultry Sci. 71: 1179-1187. ![]() Commonly
used abbreviations & terms: 1-VR Measure of the expected explained
variation Bias Systematic difference between
true and predicted values H Measure
of spectral similarity ISI Infrasoft
International (major supplier of software for NIRS) NIR Near infrared; 700-2500
nm. NIRS Near infrared reflectance spectroscopy r² Explained
variation RMSEP Root mean square error of prediction (true variability of an estimate) SEC Standard error of calibration
(variation within the reference population not explained by the calibration) SECV Standard error of cross validation (expected
variability of an estimate) RMSEP Variability of prediction error T Student
T value (statistical similarity to reference population) VIS Visible region; 400-700 nm | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||