Skip to content
Permalink
f6deeaf917
Go to file
 
 
Cannot retrieve contributors at this time
1049 lines (942 sloc) 86.7 KB
% This is based on the LLNCS.DEM the demonstration file of
% the LaTeX macro package from Springer-Verlag
% for Lecture Notes in Computer Science,
% version 2.4 for LaTeX2e as of 16. April 2010
%
% See http://www.springer.com/computer/lncs/lncs+authors?SGWID=0-40209-0-0-0
% for the full guidelines.
%
\documentclass{llncs}
% Table package needs
\usepackage{tabularx,booktabs}
\usepackage{multirow}
\usepackage[normalem]{ulem}
\usepackage[english]{babel}
% Image package needs
\usepackage{graphicx}
%\usepackage{graphics}
% Equation packages
\usepackage{amsmath}
% Graph generation package
\usepackage{tikz}
\usepackage{listings} % Include the listings-package
\usepackage{color}
\usepackage{balance}
\useunder{\uline}{\ul}{}
\definecolor{darkgreen}{rgb}{0,0.5,0}
\definecolor{mygreen}{rgb}{0,0.6,0}
\definecolor{mygray}{rgb}{0.5,0.5,0.5}
\definecolor{mymauve}{rgb}{0.58,0,0.82}
\lstset{ %
backgroundcolor=\color{white}, % choose the background color; you must add \usepackage{color} or \usepackage{xcolor}
basicstyle=\ttfamily\scriptsize, % the size of the fonts that are used for the code
breakatwhitespace=false, % sets if automatic breaks should only happen at whitespace
breaklines=true, % sets automatic line breaking
captionpos=b, % sets the caption-position to bottom
commentstyle=\color{mygreen}, % comment style
deletekeywords={...}, % if you want to delete keywords from the given language
escapeinside={\%*}{*)}, % if you want to add LaTeX within your code
extendedchars=true, % lets you use non-ASCII characters; for 8-bits encodings only, does not work with UTF-8
frame=single, % adds a frame around the code
keepspaces=true, % keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
keywordstyle=\color{blue}, % keyword style
% language=C, % the language of the code
morecomment=[l]{--},
morekeywords={property,set,is,type, constant, enumeration, end, applies, to, inherit, of, *,...}, % if you want to add more keywords to the set
numbers=left, % where to put the line-numbers; possible values are (none, left, right)
numbersep=5pt, % how far the line-numbers are from the code
numberstyle=\tiny\color{mygray}, % the style that is used for the line-numbers
rulecolor=\color{black}, % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. comments (green here))
showspaces=false, % show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
showstringspaces=false, % underline spaces within strings only
showtabs=false, % show tabs within strings adding particular underscores
stepnumber=1, % the step between two line-numbers. If it's 1, each line will be numbered
stringstyle=\color{mymauve}, % string literal style
tabsize=2, % sets default tabsize to 2 spaces
title=\lstname % show the filename of files included with \lstinputlisting; also try caption instead of title
}
\begin{document}
\title{An Adversarial Risk-based Approach to Embedded Systems Security Modeling and Design}
%
\titlerunning{Adversarial Risk-based Security Modeling} % abbreviated title (for running head)
% also used for the TOC unless
% \toctitle is used
%
%\author{Paul Wortman \and John A. Chandy}
%
%\authorrunning{Ivar Ekeland et al.} % abbreviated author list (for running head)
%
%%%% list of authors for the TOC (use if author list has to be modified)
%\tocauthor{Ivar Ekeland, Roger Temam, Jeffrey Dean, David Grove,
%Craig Chambers, Kim B. Bruce, and Elisa Bertino}
%
%\institute{University of Connecticut, Storrs CT 06269, USA}%\\
%\email{I.Ekeland@princeton.edu},\\ WWW home page:
%\texttt{http://users/\homedir iekeland/web/welcome.html}
%\and
%Universit\'{e} de Paris-Sud,
%Laboratoire d'Analyse Num\'{e}rique, B\^{a}timent 425,\\
%F-91405 Orsay Cedex, France}
\maketitle % typeset the title of the contribution
\begin{abstract}
%AADL is a common use language that has been developed and tweaked over the years to allow the ability to
%describe model behavior and specifications, with more recent attempts to define language for security
%requirements and verification. This paper examines previous implementations of behavior, requirements, and
%security in AADL and then goes to propose a new framework for better integration and description of security
%requirements and behavior within the AADL lexicon.
Embedded systems design and verification has become more complicated as aspects of security need to be included
in the design process. The obvious implications of adding security are the need to account for impacts of
loss (risk) and accounting for the ensuing increased design costs. The considerations that are not
traditionally examined are those of the adversary and the defender of a given system. Without accounting for
the view point of the individuals interacting with any secure embedded system design, one can not verify and
select the most advantageous security design.
\keywords{security modeling, security framework, secure system design}
\end{abstract}
\section{Introduction}
\label{sec:intro}
%AADL, like most modeling languages, must be able to describe not only the requirements of a system, but also the constraints, capabilities, and costs of various implementations and methods for the purpose of modeling a gamut of different designs. Coupling this already large space with security causes the considerations and influences on the problem to grow considerably. This paper will choose to examine the field of modeling embedded system security through the lens of the Architecture Analysis \& Design Language (AADL).
Modeling security risk for use in systems design is a difficult problem that has not been thoroughly explored. To properly model security one has to account not only for the security requirements being imposed by a user, or organization, but also must account for the architectural components and their capabilities when designing a best-fit solution to a given security concern. The security requirements can range from such vague concepts as ``my data must remain secure'' to more concrete requirements of ``this specific communication standard must be used''. Each requirement is capable of being implemented in a variety of manners and methods. These differences are further defined by the architectural components and their capabilities. Elements ranging from time spent to complete a given task, power consumption rate, heat radiated over time, and size, or area, that a given component will require on a printed circuit board (PCB). To further complicate matters, one must take these opposing aspects of the system design process, represent them using meaningful metrics that can be calculated from some deterministic information, and then compare and contrast generated solutions for implementing the most favorable variation of produced embedded systems security model. Fortunately there are
methodologies and techniques %(e.g. Platform Based Design)
that aid in the development and improvement of security modeling approaches. For example, Platform-based design~\cite{Vincentelli2007} is a prime example of how one can take the functional space (including security requirements) and the architectural space (components and capabilities) and develop a mapping function that can produce solutions to a given design problem.
%As shown in Figure~\ref{fig:recursivePBD},
One can then take the mapped solution and use this as the new functional (or architectural) space for the next iteration of solution mapping.
%\begin{figure}
% \includegraphics[height=6.5cm,width=\textwidth]{./images/recursivePBD.png}
% \caption{Visualization of Recursive PBD Model~\cite{Vincentelli2007}}
% \label{fig:recursivePBD}
%\end{figure}
In order to use a design process such as this, we need to be able to have metrics that are able to evaluate the quality of a design solution in relation to the quality of other possible design solutions.
How does one begin to place metrics on an arbitrary measure like security risk? Risk is generally treated as a probability that an event will occur. To understand risk from an embedded systems security modeling standpoint, one must first determine how to define risk in a meaningful manner, and how to then apply this metric to the situation that involved said `security risk' (more on this topic will be explored in Section~\ref{sec:riskDefinition}). Quantitative aspects of security will be relatively simple to incorporate with risk calculations. How does one define qualitative aspects of security? One method that can be used to assign a quantitative value to a qualitative property is to use a relative ranking system to scale all available
solutions from a 0.00 -{}- 1.00 scale as explored by Ferrante et.~al.~\cite{Ferrante2013}.
First, one needs to develop a verification and selection process for taking all of the cataloged information and possible solutions \& compare and contrast solutions that meet user-defined security requirements while categorizing solutions that are maintaining within external constraints and the capabilities of the architectural components being used to produce a best fit; measured by this paper's verification and selection process (Section~\ref{sec:riskDefinition}). To begin this approach, one will first need to define the constraints of the system, describe the considerations that must be taken into account, and standardize a method by which an individual can produce a comparable metric for generated embedded system security model designs (Seciton~\ref{sec:framework}). In the following Sections, we will produce examples of the verification and selection process (Seciton~\ref{sec:simpleExample}), then expand the discussion to include additional considerations (e.g. physical, adversarial). Lastly, we will speak to some last, larger, considerations followed by the conclusions and future work towards producing an adversarial risk-based approach to embedded systems security modeling.
%\section{Motivation and Related Work}
%Work has been made to describe behavior, errors, some security properties in terms of verification and validation within the AADL language. These developments have occurred in a variety of annexes, most recently the security annex extensions and alterations that have been occurring throughout the summer of 2016~\cite{AADLSecAnnex,AADLSecAnalysis}. Attempts are always being made to improve AADL and incorporate more in the description and detail of the language. Recent work includes the definition of specific security-based properties, ranging from `AccessProteciton' to security level definitions to even such extensive work as defining all the different aspects of encryption (shown in Listing~\ref{lst:AADLSecEncryption}).
%
%\begin{lstlisting}[caption={AADL Security Annex Definitions of Encryption~\cite{AADLSecAnnex}},label={lst:AADLSecEncryption}]
%encryption : security_properties::encryption_type applies to
% {port, virtual bus, bus, memory, access};
%encryption_type : type record (
% method : security_properties::supported_encryption_method;
% algorithm : security properties::supported_encryption_algorithm;
% public_key : aadlstring;
% private_key : aadlstring;
% key : aadlstring;
% operation_mode : security_properties::supported_operation_mode;
%};
%supported_encryption_method :
% type enumeration (symmetric, asymmetric, clear);
%supported_encryption)algorithm :
% type enumeration (tripledes, des, rsa, blowfish, aes, clear);
%supported_operations_mode :
% type enumeration (ecb, cbc, pcbc, cfb, ofb, ctr);
%\end{lstlisting}
%Current language standards used to describe security concepts, requirements, and constraints has not been developed well enough to be `all-encompassing'. Recent AADL extensions to the security annex describes the security aspect of `trust' as a non-binary value, which this paper sees has not an accurate reflection of security concerns and does not allow for accurate verification and validation of security requirements and behavior. While the concept of `trust' can be influenced by aspects of reliability, the metric used for measuring trust should be binary as one can not say that they trust a component, or system, 80\% of the time; whereas reliability can be easily described as a percentage.
With all the work towards describing and defining security modeling aspects, there is a need to develop a method for calculating `estimation metrics' that can be used to compare and contrast generated solutions. To even begin development of these metrics, one needs to be able to account for a `security metric' that can represent differing solutions, algorithms, methods for tackling security concerns and requirements.
%This `security metric' value should be `relatively deterministic'. What this paper means it that the values calculated should be deterministic, but that this deterministic nature will originate from a scaling, or ranking, that is `relative' to the capabilities of the security aspect being examined (e.g. ranking encryption techniques~\cite{Ferrante2013}). Further more,
These metrics must eventually all have the same basic `unit of measure', thus allowing for a relevant interpretation of developed metrics and calculated values for various security solution implementation designs.
%The easiest is a monetary amount (USD). Since everything carries some weight of time (development, testing, production, design), then at some point one will need to convert a unit-less metric to a time metric to a monetary metric.
To better describe these security models, since all aspects carry some weight of time (development, testing, production, design), it would be advantageous to have the final estimation metric represented with a monetary
% over time
unit of measurement.
% Add in an image of the drawing from the bulletjournal on the process
\begin{figure}
\centering\includegraphics[height=6cm,width=0.8\textwidth]{./images/aadl_security_framework.png}
\caption{Visualization of Security Design Framework}
\label{fig:AADLSecFrame}
\end{figure}
%Section~\ref{sec:framework} will propose a security design framework that can be used for applying a
%verification and selection process by which a set of risk-based equations can be used to assign metrics to
%developed embedded system security models.
The proposed design framework in Figure~\ref{fig:AADLSecFrame} would
require the following steps to take place:
\begin{enumerate}
\item Creation of a low-level component library that would contain normal and secure version implementations of each base component within the architectural space used for model generation.
\item Formalized description and definition of higher level security requirements that may come from user-defined needs or from the experience of knowledge of security experts.
\item Creation of a mapping process by which security requirements and secure component specifications can be uniformly verified and selected to allow for the generation of potential secure architectural system model solutions to the given inputs.
\item Verification tools to validate mapping implementation solutions.
\end{enumerate}
Other additional aspects of this framework, that could come from the existing tools, extensions, and annexes would include code generated using the secure models.
%Work towards developing these sorts of tools for secure architectures (e.g. seL4) is already one of the focuses of current AADL security annex work~\cite{AADLSecAnalysis}.
While this framework is a vision of the overall design process, in this paper, we focus on just the requirements
modeling and the verification process. Specifically, we focus on how to model security goals and requirements
in such a way that they can be measured and validated. Ideally one desires a metric that incorporates not only
the impact of design decisions on a system and the cost of producing a design, but also account for the
examination and evaluation an attacker may make of a system. However, before one can even begin to apply a
framework to this verification and selection process, one needs to first be able to define `Security Risk' so
that a relatively deterministic formula can be used to obtain a meaningful metric.
\section{Related Work}
In this section, the paper examines the traditional definition of risk followed by a brief explanation of the work by Ferrante et.~al. in preparation for the proposition of defining security risk.
\subsection{Traditional Risk}
\label{sec:traditionalRisk}
% Risk traditionally defined
Risk is generally defined as the potential of gaining or losing something of value. Value can be seen as physical health, emotional well-being, financial wealth, etc. Another definition of risk involves viewing risk
as an intentional interaction made with some uncertainty. In this scenario, uncertainty is defined as a potential, unpredictable, and uncontrollable outcome; risk is seen as a consequence of action taken in spite of some given uncertainty. Traditionally, risk is a consideration of losses based on the probability of a given event will occur~\cite{Mukhopadhyay2013,pal2014will,fauntleroy2015cyber,biener2015insurability,armando2004satmc,brucker2012securebpmn,gonzalez2012quantitative,mukherjee2013attributed,marotta2015survey}. Depending on the point-of-view of the individual measuring risk, its definition and application can vary a significant amount.
%For example, risk can be the analysis of expected loss %(as shown in Equation~\ref{equ:expectedLoss}).
%Risk is not a certainty of an event occurring, but a probability that it will happen.
But to develop an equation for risk one must first define the potential of events and the losses that could be incurred. Possibility, in risk, depends on two aspects: (1) threat and (2) vulnerability ~\cite{Mukhopadhyay2013}. Threat is defined as the cause of risk (e.g. fire, kidnapping, leakage of sensitive information, etc.). Vulnerability is defined as the existing flaw or weakness which can be exploited and result in an accident. The concept of risk states that risk may result in losses for an agent, user, company, etc. Losses occur because of the consequences of an accident (defined as Impact). Depending on the impacted asset, `Impact' may be defined as a tangible (e.g. loss of revenue or financial penalties) or as intangible (e.g. loss of productivity or loss of reputation)~\cite{Mukhopadhyay2013}. An `asset' can be defined as anything valuable to a user or organization or company. An asset can be (1) a physical object, (2) secrete information, (3) business goal, etc. As mentioned earlier, risk requires an element of probability, meaning that the probability value acts as a 0.00 -{}- 1.00 scale weight. Putting everything together, risk is generally represented as follows:
\begin{equation} \label{equ:riskDefinition}
Risk = Probability * Impact
\end{equation}
% Define Risk for this paper
For the purpose of this paper, risk will be represented as a combination of probability and impact (as shown in Equation~\ref{equ:riskDefinition}).
%The reason for this interpretation is that from a security lens, it is much easier to quantify probability and impact.
In addition to these, we have developed a methodology for evaluating design choices based on impact (risk), cost of the design, and also incorporating the attacker's side of examining any given embedded system. Without a proper method for defining risk and cost models, one can not verify, evaluate, compare, and contrast designs in a relevant and worthwhile manner.
% Differences can occur when assessing risk depending on the point of view of the individual
%Summarizing, risk is a combination of (1) a threat, (2) a vulnerability, (3) an impact.
Complications in
security risk identification can come from a lack of experience and standards or due to the evolution of a system. The first comes the fact that defining `security risk' is still novel and does not have standardized procedures for dealing within the cyber-modeling domain. Secondly, the system within an organization can change quickly with new technologies appearing very often; changing the landscape of `cyber risks' and other cyber-domain concerns.
%Risk can be assessed differently based on how it is examined.
Depending on where uncertainties originate from, how impacts of actions are measured, and how these variables interact with each other, different variations of risk equations can be developed. To further complicate possible risk calculations, the equation for interpreting risk can differ based on the role of the individual measuring said risk. What may be a calculated risk to a defender, could prove to be advantageous to an attacker by causing less risk of being observed or even allow for
less risky attacks to be performed against a system. Another example would be that heavy security
implementation may cause a larger risk value for an attacker, but would produce a minimal risk value for the
defender. Further examination of attack and defense considerations will be continued in
Section~\ref{sec:attackDefense}.
%\begin{equation} %\cite{Mukhopadhyay2013}
%\label{equ:expectedLoss}
% Expected Loss = Event Frequency * Risk Amount
%\end{equation}
%
%One can measure risk from the probability of a failure of a given component (e.g. firewall, anti-virus, both), the loss amount for each component failure (e.g. firewall, anti-virus, both, none), and the expected loss (average loss)~\cite{Mukhopadhyay2013}. In this manner an individual can measure risk for a larger, interconnected system, but as the scope of the risk examination changes, so does do the methods by which risk is measured.
\subsection{Quantitative and Qualitative Security}
% Summary and application of the Ferrante work
% Development of the verification process
To begin developing the verification and selection process, one needs to create a metric of weights based on the relative importance of any given solution $s_i$ in comparison to another solution $s_j$~\cite{Ferrante2013}. The purpose of these weights is to allow for comparison of varying solution elements, relative to each other, as they are used within embedded system security modeling solutions. Importance is defined as a ranking that will depend on the case of implementation by a given company/group. Because this value is dependent on the `source' examining a particular security model, the value will be partially arbitrary and partially cost analysis. An other influencing factor could also come from the preference/wheel house of a given company or individual.
% Explain the work done by Ferrante et. al.
Borrowing from the work of Ferrante et.~al.~using the Analytical Hierarchy Process (AHP), we examine the process developed for producing weights for various security solutions.
%First, one must create an arbitrary 1-{}-10 ranking of the importance of any given element in comparison to any other elements of the security design model; where 1 is defined that both elements are equally important and 10 is defined that one element is absolutely more important than the other. These values are then inversed to create a matrix of `relative importances' of the various security model elements with respect to each and every other element in the design. Since the elements of the matrix are based on human judgement there is the possibility that inconsistency may exists. Ferrante et.~al.~ did develop equations to recognize these inconsistencies, but further examination of the process is left to the reader. However, not all aspects of security are going to be quantitative, some will be more qualitative and thus more difficult to deterministically produce values for. A rough method of producing quantitative values from qualitative properties was also proposed by Ferrante et.~al.~; by using the weight developed for a given requirement and multiplying this value by a series of binary representations of whether or not a given feature meets the same requirement that the feature in question is attempting to meet. This is represented by the following equation from Ferrante et.~al.'s work~\cite{Ferrante2013}:
%
%\begin{equation} \label{equ:qualitative}
% L_i = \sum_{j=1}^{R} g_j * v_{ji}, i = 1,2,...,S
%\end{equation}
One must note that there are essentially two values being spoken about in the work by Ferrante et.~al.; first is the generation of a security metric (named security level) and second generation of a weight metric. The creation of the `security level' metric is a simple mapping of some physical property to a 0.00 -{}- 1.00 scale with 1 being the highest possible security level. Once one has created these weight ($w$) and security level ($SL$) metrics, these values can be used to produce security metrics about a given embedded systems security model design.
% Introducing the Security Metric equation
The next consideration is how does one take these produced values and develop a meaningful equation for generating a comparable metric. Following the examination of the work by Ferrante et.~al.~, their definition of `security metric' is as follows:
\begin{multline} \label{equ:securityMetric}
Security Metric (SM) = \\
w_{elem1} * SL_{elem1} + w_{elem2} * SL_{elem2} + {...} + w_{elemN} * SL_{elemN}
\end{multline}
Each `element', in Equation~\ref{equ:securityMetric}, can describe different algorithms, elements, properties, and capabilities of the different components of a system or device.
%It is worth noting that this equation is not necessarily limited to a single use within entire process.
This same style can be used at higher level abstractions of the same design to produce an overall security metric representing a larger system of components. Harking back to our discussion of risk, the following is an example of calculating aggregated security metrics of a system (i.e. network security):
\begin{multline} \label{equ:overallSecMet}
Overall SM = \\
w_{anti-virus} * SM_{anti-virus} + w_{firewall} * SM_{firewall} + {...} + w_{element i} * SM_{element i}
\end{multline}
% Introduction to paper's Security Risk security
This allows for a designer to now create some arbitrary metric to represent whether or not a given design is better or worse at meeting not only the requirements imposed but also in comparison to other available solutions to the same design problem. While this is a step in the right direction, there no units attached to the overall
security metrics produced by this work. Without a proper unit attached to a generated metric, the entire process remains arbitrary and more difficult to use in a relevant and meaningful manner. %Calling back to the equation proposed in Section~\ref{sec:riskDefinition}, allow us
We make use of the same techniques Ferrante et.~al.~developed but apply them in a more meaningful, cost-based
manner. Thus, instead of focusing on an arbitrary security metric (SM), we focus instead on the security level
(SL) which can serve as a useful approximation of the probability that an attacker can defeat a particular
security feature. In this way it is possible for a `security risk' metric to be generated but also carry a
recognizable worth. %instead of just an arbitrary number scheme.
However, these values of cost must come from somewhere, and that is the purpose of the impact variable. This is because the impact is a representation of the potential loss caused by an event.
% Trouble when incorporating security into risk calculations
Different methods by which security can be incorporated into risk management include: as a weight representing implementation of security solutions, as a probability that a security concern is met or attacked, the
possibility of a security failure, etc. Security levels can also be interdependent depending on implementation and scenario/situation.
%Given all these variations, how can one define security in a meaningful and relatively deterministic manner? The first part that needs to be tackled is why is `relatively' good enough for this security metric? The reason that
A degree of `relativity' is required for developing a security metric, because security changes and evolves over time, meaning that the algorithms and methodologies will transform and improve over time. This change means that the measurement for security must remain important relative to the existing security solution space.
%The reason that this value can not be fully deterministic is that security is constantly progressing and improving, meaning that any deterministic nature in the calculation of a security metric must also account for this change over time.
Ideally embedded system security modeling should have more deterministic interpretations of generated security solutions, but at the current point this is not realistically achievable.
%The purpose of this paper is to propose a method for combining security and risk in a measurable and meaningful manner. Taking the already defined risk equation (i.e. Equation~\ref{equ:riskDefinition}), we move to add in the existence of a `security metric' value ($SM$) and impact to the probability that either a direct or an indirect attack occurs to a given system.
%
%\begin{equation} \label{equ:securityRisk}
% Security Risk (SR) = \frac{p_{da} * i}{SM}
% + \frac{p_{ida} * i}{SM}
%\end{equation}
%
%This can be seen in Equation~\ref{equ:securityRisk}, where the probability aspect of risk is split between the
%chance of how exactly an attack will occur. $i$ is the impact, i.e. the cost impact of a event occurring. %XXX is this the right definition of cost weight?
%$p_{da}$ represents the probability of a direct attack, where direct attack is defined as an event where an
%attacker directly attempts to brute force a given security mechanism or standard. $p_{ida}$ represents the
%probability of an indirect attack, where an indirect attacker is one where a malicious user attempts to circumvent existing security by some aspect that is not directly related to the mentioned security implementation.
%%XXX What is the definition of security metric in the equation?
%Once risk has been defined in the scope of examination, one can develop an `Estimation Metric' that can be compared and contrasted with each other to determine the `worth' of any given design. However, before these metrics can be developed, one must first determine a framework by which these calculations will be incorporated to allow for a relevant and meaningful interpretation of verification and selection metrics.
\section{Design Oriented Examination of Risk}
\label{sec:riskDefinition}
While traditional views of risk only deal with a single source of probability, the examination of security risk
is more involved due to multiple sources of probability.
% (e.g. probability of attack, probability of success).
Each possible attack vector upon a system is dependent on a series of exploits, meaning that as multiple steps
are taken to perform a successful data exfiltration attack, one needs to accurately aggregate the individual
costs of each event in the process. To help with the visualization of potential attack vectors, we
present an illustrative example of a potential `attack tree' on a piece of data.
\begin{figure}
\centering
\begin{tikzpicture} [xscale=3, yscale=1]
%\draw [<->] (0,3) -- (0,0) -- (4,0);
\node at (0.5,3) {MiTM};
\draw [yellow] (0.3,3.2) rectangle (0.7,2.8);
\draw [->] (0.5,2.8) -- (1,2.25);
\node at (1.5,3) {Brute Force};
\draw [yellow] (1.2,3.2) rectangle (1.8,2.8);
\draw [->] (1.5,2.8) -- (1,2.25);
\node at (2.5,3) {Social Engineering};
\draw [yellow] (2.0,3.2) rectangle (3.0,2.8);
\draw [->] (2.5,2.8) -- (3,2.25);
\node at (3.5,3) {SW Vuln};
\draw [yellow] (3.25,3.2) rectangle (3.75,2.8);
\draw [->] (3.5,2.8) -- (3,2.25);
\node at (1,2) {Break Encryption};
\draw [orange] (0.55,2.25) rectangle (1.45,1.85);
\draw [->] (1,1.85) -- (2,1.2);
\node at (3,2) {Root Access};
\draw [orange] (2.7,2.25) rectangle (3.3,1.85);
\draw [->] (3,1.85) -- (2,1.2);
\node at (2,1) {data};
\draw [red] (1.8,1.2) rectangle (2.2,0.8);
\end{tikzpicture}
\caption{Illustrative Example of Attack Vector Tree}
\label{fig:attackTree}
\end{figure}
Each of the events illustrated in Figure~\ref{fig:attackTree} has it's own event probability and cost.% attached to the respective event.
Further complication in the attack vector tree can come from variation in each event - for example, is the encryption being attacked 64-bit or 128-bit?
\begin{figure}
\centering
\begin{tikzpicture} [xscale=3.5, yscale=2]
\draw [<->] (0,1.5) -- (0,0) -- (3,0);
\node [below right] at (3,0) {$p_s*A$};
\node [left] at (0,1.5) {$p_a$};
\draw [dashed, gray] (0,1) -- (3,1);
\draw (1,0) to [out=90,in=180] (3,1);
%\draw [green, domain=0:3] plot (\x, {1 - exp(-\x - 1)});
\draw (1,-0.1) -- (1,0.1);
\node [below] at (1,0) {$c_a$};
\node [left] at (0,0) {0};
\node [left] at (0,1) {1};
\end{tikzpicture}
\caption{Estimation of Attack Risk Behavior}
\label{fig:attackRisk}
\end{figure}
Beyond the notion of an attack vector, the traditional view of risk also does not account for an attacker's
motivations. This paper moves to develop a model that will not only account for the fact that the calculation is reliant upon multiple sources of probability but also represent risk due to an attacker's intent.
First, we modify the traditional risk model (Eq.~\ref{equ:riskDefinition}) to represent the security risk, $SR$, as a combination of the probability of an attack ($p_a$), probability of the attack succeeding ($p_s$), and the impact of the attack.
\begin{equation} \label{equ:attackRisk}
SR = p_a * p_s * I
\end{equation}
The probability of an attack is defined as the chance that a given system, or element, is going to be targeted by an attacker's efforts. In other words, if the system is not of any interest to an attacker, i.e. $p_a=0$, the risk is 0. If the system is highly desirable to attackers, i.e. $p_a=1$, the risk is dependent completely on the probability of success. The probability of success is defined as the chance that an attacker's efforts will pay off positively for the attack, or negatively for the defending system. The probability of success is thus tied to the quality of the security defenses built into the system design. While determining this probability is difficult and will require further study, a useful approximation can be represented by the security level (SL) value discussed earlier. This alters Equation~\ref{equ:attackRisk} to become:
\begin{equation} \label{equ:attackRisk2}
SR = p_a * (1 - SL) * I
\end{equation}
Now, we turn our attention to determining an appropriate value for $p_a$. First, we introduce a new variable,
$A$, which we define as the value of an attack to an attacker. This is distinct from $I$ which is the impact of
an attack to the defender. As an example, a credit card number has a current value of a few dollars to an
attacker. However, a defender may have impact costs of hundreds of dollars due to lost reputation, credit
monitoring fees, regulatory fines, etc. Similarly, different attackers may also assign different values, $A$,
to assets under attack. For our purposes, we assume a single attacker model, but one could arrived at an
aggregate $A$ with weighted assignments due to different attackers. First, we note that as $p_s*A$, which is
the probability of success multiplied by the value to an attacker, changes so should the value of $p_a$, or the
probability of an attack. In other words, one would expect that as the expected value of an attack increases,
the likelihood of an attack will also increase. It is also worth noting that $p_a$ is influenced by the cost to
attack ($c_a$) a given system, in such a manner that if $p_s*A < c_a$ then $p_a$ = 0\%; otherwise if $p_s*A \geq
c_a$ then $p_a$ should grow from 0\% to eventually 100\%. This is because $p_a$ should be zero until such a
time that the $p_s*A$ aspect become meaningful. The relationship between the expected attack
value and $p_a$ is not well defined and further study is needed to determine an accurate model. As a simple model, we suggest an exponential relationship as shown in Figure~\ref{fig:attackRisk} and the following equation:
%Harking back to Figure~\ref{fig:attackRisk}, we arrive at the equation for the probability of attack becomes:
\begin{equation}
p_a =
\begin{cases}
1 - e^{-\alpha(p_s*A - c_a)},& \quad\text{if } p_s*A \geq c_a \\
0,& \quad\text{otherwise}
\end{cases}
\end{equation}
where $\alpha$ captures the sensitivity of $p_a$ to the expected attack value. In our example in Section~\ref{sec:simpleExample}, we assume that $\alpha=1$. Determination of appropriate values of $\alpha$ is dependent on the particulars of an attack scenario.
Incorporating into Eq.~\ref{equ:attackRisk}, the equation for determining security risk becomes:
\begin{equation} \label{equ:expandedRisk}
SR = (1 - e^{-\alpha(p_s*A - c_a)})*p_s*I
\end{equation}
One then takes Equation~\ref{equ:expandedRisk} and now accounts for an aggregation of an asset $i$ and attack vector $j$ to create the equation:
\begin{equation}
{SR}_i = \sum\limits_{j=0}^K (1 - e^{-\alpha(p_{s_{ij}}*A_i - c_{a_{ij}})})*p_{s_{ij}}*I_i
\end{equation}
where K is all the contributing attack vectors for a given asset. Using the substitution for $p_s$ in Eq.~\ref{equ:attackRisk2}, we get
\begin{equation} \label{equ:aggregatedRisk}
{SR}_i = \sum\limits_{j=0}^K (1 - e^{-\alpha((1-SL_{ij})*A_i - c_{a_{ij}})})*(1-SL_{ij})*I_i
\end{equation}
Thus, Equation~\ref{equ:aggregatedRisk} represents the aggregation of all contributing attack vectors towards the potential compromise of a specific
system asset. This security risk metric, essentially, represents the expected security cost of a device over a lifetime.
\section{Incorporating Risk into Design}
\label{sec:framework}
Now that we have presented a meaningful manner of representing security risk as a metric, we move to show that
how to use this risk metric in embedded system design.
%What does the proposed framework bring to the table that lacked in previous security framework designs?
In Section~\ref{sec:intro}, we introduced a security design framework that allows for generation of embedded system security models by assuming that the security requirements and architectural component capabilities can be represented in a quantitative manner. With the framework, shown in Figure~\ref{fig:AADLSecFrame}, one can take the risk equations developed in Section~\ref{sec:riskDefinition} and produce a meaningful metric estimation that can be used to compare and contrast varying security model solutions. Combination of the framework's verification process and the risk equations developed is done in the following manner:
\begin{enumerate}
\item Define the security requirements and architectural components in a numerically meaningful manner.
\item Take these unit-less metrics and apply a `conversion equation' to them in order to produce a meaningful, `monetary unit' based metric.
\item Develop a method of interpreting the metrics in terms of requirements met, not met, and additional features added to the system.
\item Combine these metrics in a unified equation that can be used to take in values from the functional and architectural space to produce a series of embedded system security model solution metrics.
\end{enumerate}
One complication of this technique is the requirement to create `libraries' for certain aspects of the security modeling framework; for example, the various solution implementations for `keeping data secure'. More will be touched upon this subject in Section~\ref{sec:additionalConcerns}. Another concern is how one ranks all the various security solutions, algorithms, and methodologies in such a manner that the capabilities of the architectural components play a part in determining the best solution models that can be generated from a given set of constraints.
% Considerations about the verification process
Ideally verification of a design should be done through validation of the requirements that were met, not met, and any additional features that were introduced due to design decisions of the developer. Certain aspects of this verification process are easier to tackle but just have not had the time and effort focused on them. One such aspect is defining security terms in a manner that can be standardized for use. However, although choosing the language to describe security concepts and ideas may be simple, having that language remain flexible and relevant over the course of security's evolution is not as easy. Working towards an effective, rigorous, and standardized security modeling framework verification methodology is something that will not only benefit the security community, but will allow for better representation and understanding in other fields as well (e.g. business).
%\begin{multline} \label{equ:securityRisk}
% `Security Risk' = Security Level * direct attacker probability * cost weight \\
% + Security Level * indirect attacker probability * cost weight
%\end{multline}
% Introduce paper's Estimation Metric equation
%While having an equation for security risk is great, this does not allow for a representation of a system as a whole.
Additional aspects that must be taken into account include costs of implementing a given solution, the cost of maintaining a given solution,
%how a generated solution's ranking will change based on the user type interacting with the system,
operational costs of a given solution.
%, as well as determination of the number of requirements met, or not met, by any chosen design solution.
Taking these aspects into account, we propose the following equation to calculate an overall cost
% and `estimation metric'
for any produced embedded system security modeling design.
% Note: Removed the c_{si} element from here to be used/referenced later for the comparison graphing
\begin{equation} \label{equ:cost}
Costs = c_i + c_m + c_o %+ c_{si}
\end{equation}
%\begin{equation} \label{equ:estimationMetric}
% Estimation Metric = SR+Costs
%\end{equation}
Some of the values for Equation~\ref{equ:cost}, implementation and maintenance costs ($c_i$ and $c_m$ respectively), are expected to be flat costs that are pre-calculated by a company or business since these values will be specific to the given organization. An important difference between these values are that the $c_m$ and $c_i$ values do not incorporate the operational costs of the design, they only account for the cost of initial implementation of a system design and the cost of performing upkeep for said design.
The operational cost ($c_o$) is a value coming from a given individual or company; based on a monetary cost over time unit of measurement.
%In Equation~\ref{equ:estimationMetric}, the `requirements weight' ($w_r$) variable can be easily constructed using the same techniques developed by the Ferrante et.~al.~ to compare the requirements and features brought to a design due to the methods, components, and algorithms implemented in a given design.
%%To simplify calculations, one can stick to a more binary representation of requirements as either being met or not being met.
%The behavior of this requirements weight should be such that having `negative requirements' (e.g. requirements not being met) should cause a larger cost of the design due to specific needs not being met; hence the purpose behind inverting the value.
%The aspiration is that this represents that operation cost of the design being weighted by the number of requirements that are met and those that have not been met (using a 0.00 -{}- 1.00 scale).
%Now that aspects of this framework's verification and selection process have been explained, allow us to apply these techniques to a sample example.
Now that two equations have been proposed, one for security risk and one for costs, we move to show how this information can be used for a framework verification and selection process by applying these techniques to a simple example.
\section{Exploring a Simple Implementation}
\label{sec:simpleExample}
We now introduce a simple example of a verification and selection process to produce a meaningful metric that can be used to compare design decisions. The purpose of this examination is to give a concrete example of the verification equations being used to rank a set of produced solutions to a chosen design problem, as well as illustrate how one may go about developing some of the required rankings. When initially designing a system, there are a large number of considerations that must be taken into account. From power consumption requirements, architectural needs, spacing of parts, and weight of the developed device. When incorporating security elements, the problem becomes larger.
%For example, allow this paper to assume that the elements in Table~\ref{elementTypes} have numerous variations that can be simplified into a few types.
% Add in the table of elements and variations
%\begin{table*}[]
%\centering
%\caption{Table illustrating different component variations}
%\label{elementTypes}
%\begin{tabular}{@{}llllll@{}}
%\toprule
%\multicolumn{6}{c}{Component Types Table} \\ \midrule
%\multicolumn{1}{c}{\multirow{2}{*}{{\underline{\textbf{Elements}}}}} & \multicolumn{5}{c}{\textbf{Types}} \\
%\multicolumn{1}{c}{} & \multicolumn{1}{c}{{\underline{\textbf{I}}}} & \multicolumn{1}{c}{{\underline{ \textbf{II}}}} & \multicolumn{1}{c}{{\underline{\textbf{III}}}} & \multicolumn{1}{c}{{\underline{\textbf{IV}}}} & \multicolumn{1}{c}{{\underline{ \textbf{V}}}} \\
%Memory & Unprotected & Protected & Encrypted & Obfuscated & Combo \\
%Bus & Unprotected & Encrypted & Non-sniffable & & \\
%Processor & Simple & Embedded Encryption & Pure Encryption & & \\
%Data & Plain-text & Encryption & Protected & & \\
%Port & Normal & Encrypted & Protected & & \\ \bottomrule
%\end{tabular}
%\end{table*}
%This table will represent the possible architectural components choices that can be made when attempting to develop a new security embedded systems design.
The simple example that we will examine is that of a wireless transmitter.
%Listing~\ref{lst:AADLUserDefineLow} shows the AADL user-defined description of the wireless transmitter element being discussed. This paper makes use of AADL to aid in modeling embedded systems with security properties and constraints for giving a practical example of our proposed adversarial risk-based approach to embedded systems security modeling. AADL has a large and active community that works towards development and improvement of new annexes or other extensions to the existing AADL lexicon. In addition to the ease of extensibility, AADL is an easy language to pick up and learn. This paired with the way in which AADL has been intuitively built, allows for ease of extension by other users or even the community at large.~\cite{AADLV2Overview}
%
%\begin{lstlisting}[caption={Example of User-defined Lower Level Components},label={lst:AADLUserDefineLow}]
%system implementation transmitter.encrypt_i
% -- Subcomponents of the transmitter
% subcomponents
% ant_in : system recv_antenna.normal_i;
% ant_out : system trns_antenna.encrypt_i;
% encrproc : processor procbase.encryptembedded_i;
% -- Connection definitions of the transmitter
% connections
% c0 : port ant_in.wired_out -> encrproc.input_port;
% c1 : port encrproc.output_port -> ant_out.wired_in;
% -- Flow path definition for the transmitter
% flows
% f0 : end to end flow ant_in.f0 -> c0 -> encrproc -> c1 -> ant_out.f0;
% -- Additional properties of the transmitter
% properties
% securityspecs::has_encryption => true;
%end transmitter.encrypt_i;
%
%processor implementation procbase.encryptembedded_i
% properties
% securityspecs::has_encryption => true;
% securityspecs::encryptmodule_class => embedded;
% securityspecs::encryption_class => AES;
% securityspecs::encryption_variation => b256;
% securityspecs::has_PUF => false;
% securityspecs::has_TPM => false;
% securityspecs::has_encryptedflash => false;
% securityspecs::isTamperProof => false;
%end procbase.encryptembedded_i;
%\end{lstlisting}
The basic requirements are that any data that is transmitted must be protected. For our purposes, we assume that there are three possible hardware design solutions that exist of said transmitter, as shown in Figure~\ref{fig:exampleDesigns}. We then expand upon the first hardware design to create a total of five potential solutions for the wireless transmitter design.
%\begin{enumerate}
% \item Non-encrypted communication with a separate input and output buses.
% \item Encrypted communication with separate input and output buses.
% \item Non-encrypted communication with a single IO bus.
% \item Encrypted communication with a single IO bus.
%\end{enumerate}
\begin{figure}
\centering\includegraphics[height=4.25cm]{./images/encrypted_transmitter_sidebyside.png}
\caption{Design Implementations of Wireless Transmitter}
\label{fig:exampleDesigns}
\end{figure}
The five solutions instances compose of (1) a baseline system where there is no encryption, (2) similar to design \#1 but with a software implementation of AES128, (3) same
hardware design as \#1 but with a software implementation of AES256, (4) a processor with a hardware implemented version of AES128, and (5) a processor with a hardware implementation of AES256.
%with built-in encryption, (3) a processor with built-in encryption and two communication channels, and (4) a
%basic processor with software encryption and two communication channels.
%In these design choices, we envision
%that the second channel allows for the creation of a separate pathway for encrypted data that may be difficult
%for attackers to probe. Likewise,
The processors with built-in encryption are more difficult for attackers to
probe and retrieve encryption keys, thus providing more security at some extra cost.
%, of a single possible solution being generated based on two aspects of the architectural space: (1) the number of IO buses available and (2) whether or not communication should be encrypted.
%To further simplify the requirement considerations of this example, the paper chooses to ignore the influence of IO bus variation and focus on the implementation, or lack of, encryption. In this manner, the examination goes from four variations to two variations (encryption enabled or disabled). To better pad out this encryption scenario we choose to examine the wireless transmitter under use of an optimal AES256 encryption algorithm using a MIPS processor, the `good enough' use of AES128 encryption algorithm also on MIPS architecture, and a complete lack on implementation of encryption.
%For the sake of simplicity, this paper makes use of the `relativity matrix' developed by Ferrante et.~al.~\cite{Ferrante2013} for representing the security level metrics on the encryption standards used.
%How does one create the security metric based on the given example?
The first step requires the creation of a relatively deterministic security metric value.
%The point of this is to generate a ranking for all known solutions that ranges from 0.00 -{}- 1.00 that can be used as a weight or security level with other known cost variables to produce a metric that can be used to compare potential generated solutions.
For the purpose of this paper, we will be making use of the encryption standard rankings developed by Ferrante et.~al.
The next step is how to consider differences between the implementations of the wireless transmitter.
%Differences from values can come from alternative design choices and/or algorithm and policy implementations of security or other standard constraints being imposed onto the design problem. For this simple example, the main difference examined is the implementation of encryption.
These different aspects can be compared by assigning weights for allowing relative importance, thus representing (in some arbitrary manner) the user-defined requirements imposed on the design being generated. This human-related favoritism causes the generated metric to alter from a small amount to notable change based on the chosen importance of different features.
%Other areas originate from the development of URT and $c_o$, where there are some arbitary decisions made on ranking of users or generated solutions.
Before being able to calculate out the estimation matrix for our wireless transmitter example, we make some
simplifications and assumptions to smooth the process. First we will take the assumption that there will be five separate implementations of encryption for the wireless transmitter: hardware AES256 (MIPS), hardware AES128 (MIPS), software AES256, software AES128, and no encryption. Drawing from the work by Ferrante et.~al.~, the corresponding security level (SL) values are {1.00, 0.60, 0.10} for {AES256, AES128, and no encryption}, respectively. It is worth noting that while in theory having no encryption should mean the lowest values possible (0.00) but in order to show the effect of these elements, we assume the lowest value obtainable is 0.10. Similarly, we assume the SL values for the different implementations are {1.00, 0.40, 0.10} for {hardware encryption, software encryption, and no encryption}, respectively.
%Since we have simplified the example to having a single requirement (i.e. encryption of data), then the weight value used for calculating the Security Metric (SM) is 1.00.
Now that we have SL values for the encryption algorithm and the implementation approach, we can determine an overall SL value by multiplying the two SL values because of their serial dependence. In order to arrive at an overall risk metric (SR), we must first make some assumptions about the impact ($I$), attack cost ($c_a$), and the value to an attacker ($A$). For the $I$ value we make the assumption that some company may find that the data collected by this wireless transmitter is worth \$20 if lost. % XXX is the cost weight really the impact? Should you use impact or $i$ as the variable name?
The $A$ value is assumed to be \$30 per `unit' of stolen data and the $c_a$ is assumed to be \$10 per attack attempt. Using Equation~\ref{equ:expandedRisk}, one finds that the SR values
%shown in Table~\ref{tbl:securityRisks}.
for the {AES256 HW, AES128 HW, AES256 SW, AES128 SW, None} encryption implementations are {\$0, \$6.92, \$12.00, \$15.20, \$19.80} respectively.
%% Table showing the calculated security risk
%\begin{table}[]
%\centering
%\label{tbl:securityRisks}
%\begin{tabular}{|c|c|c|c|}
%\hline
% & \multicolumn{3}{c|}{Encryption Standard} \\ \cline{2-4}
% & AES256 & AES128 & None \\ \hline
%Security Risk & 0 & 2.73 & 15.84 \\ \hline
%\end{tabular}
%\caption{Calculated Security Risk for Wireless Transmitter}
%\end{table}
From this point, we make further assumptions about the implementation cost ($c_i$),
%security implementation cost ($c_{si}$),
the maintenance cost ($c_m$), and operational cost ($c_o$) since these values would come from metrics internal to a company or organization. $c_i$ is taken to be an aggregation of the individual cost of parts.
%The costs assumed are \$3 per antenna, \$1 per IO bus, \$15 for a basic processor, \$20 for an encryption optimized processor, \$18 for processor that works with embedded technology, and \$8 for an encryption module;
\begin{table}[]
\centering
\begin{tabular}{|c|c|}
\hline
\textbf{Component} & \textbf{Cost (\$)} \\ \hline
Antenna & 3 \\ \hline
IO Bus & 1 \\ \hline
Basic Processor & 15 \\ \hline
Encryption Processor (AES128) & 20 \\ \hline
Encryption Processor (AES256) & 21 \\ \hline
%Encryption Module & 8 \\ \hline
\end{tabular}
\caption{Table of Individual Part Costs}
\label{tbl:partCosts}
\end{table}
The costs per design, using the individual costs from Table~\ref{tbl:partCosts}, are reflected in Table~\ref{tbl:calculations} for each variation of the wireless transmitter design.
%from 1-{}-4, are {19,28,24,34}, respectively.
% $c_{si}$ is taken to be \$10 for AES256 and \$6 for AES128,
$c_m$ is taken to be \$50 in drive out cost, and $c_o$ is assumed to be \$3 in operative power costs.
%per 12 hours of operation (\textbf{How to ignore time aspect?}).
%The $w_r$ value is assumed to be 1.00 if the system is encrypted and 0.10 is not; since the effect of not meeting requirements can be viewed more clearly.
%Taking the calculation of the estimation metric (EM) from Equation~\ref{equ:estimationMetric}, we produce the contents of Table~\ref{tbl:calculations} which represents the calculated %costs and security risk metric for each encryption scenario {AES256 (MIPS), AES128 (MIPS), None}.
% and how different User Risk Types (URT) also further influence the metric.
%% Please add the following required packages to your document preamble:
%\begin{table}[]
%\centering
%\label{tabl:calculations}
%\begin{tabular}{|c|c|c|c|}
%\hline
% & \multicolumn{3}{c|}{Encryption Standard} \\ \cline{2-4}
% & AES256 & AES128 & None \\ \hline
%Security Risk & 0 & 2.73 & 15.84 \\ \hline
%Costs & 113 & 109 & 103 \\ \hline
%Estimation Metric & 113 & 111.73 & 1188.40 \\ \hline
%\end{tabular}
%\caption{Calculated Values for Wireless Transmitter Example (Units USD)}
%\end{table}
%\begin{table}[]
%\centering
%\begin{tabular}{|c|c|c|c|c|}
%\hline
% & Design 1 & Design 2 & Design 3 & Design 4 \\ \hline
%Costs & \$72 & \$91 & \$88 & \$93 \\ \hline
%Security Risk & \$15.84 & \$0 & \$0 & \$2 \\ \hline
%\end{tabular}
%\caption{Calculated Values for Wireless Transmitter Example (Units USD)}
%\label{tbl:calculations}
%\end{table}
\begin{table}[]
\centering
\begin{tabular}{|c|c|c|c|c|c|}
\hline
& \textbf{Design 1} & \textbf{Design 2} & \textbf{Design 3} & \textbf{Design 4} & \textbf{Design 5} \\ \hline
\textbf{SL} & 0.01 & 0.24 & 0.40 & 0.60 & 1.00 \\ \hline
\textbf{$p_s$} & 0.99 & 0.76 & 0.60 & 0.40 & 0.00 \\ \hline
\textbf{$p_a$} & 0.99 & 0.99 & 0.99 & 0.86 & 0.00 \\ \hline
\textbf{SR} & \$19.80 & \$15.20 & \$12.00 & \$6.92 & \$0.00 \\ \hline
\textbf{Costs} & \$19.00 & \$19.00 & \$19.00 & \$24.00 & \$25.00 \\ \hline
\end{tabular}
\caption{Calculated Values for Wireless Transmitter Example}
\label{tbl:calculations}
\end{table}
From these produced numbers, a developer can apply further constraints to the design selection process by mandating that costs be kept below a certain level while minimizing security risk or vice versa. For example, a development team could compare how a growth in the impact ($I$) value changes the cost value reflected by the security risk (Equation~\ref{equ:expandedRisk}) metric. The $c_{si}$ variable represents any cost due to implementing security aspects into the design relative to the base, no encryption, design. Ideally a developer is looking for the point where as $I$ grows, the value of $SR$ should cross the $c_{si}$ threshold, thus indicating that the cost of additional security is warranted. In the case of comparing designs 1 and 5 of the wireless transmitter, one finds that the difference in costs is \$6; taken to be the $c_{si}$ threshold. Using the values for Design 1 of the wireless transmitter, one finds that the once the $I$ value becomes \$6.06 then the additional security cost of \$6, for a full security design, becomes not only feasible but favorable. An illustration of this is shown in Figure~\ref{fig:SRvI}.
\begin{figure}
\centering
\begin{tikzpicture}[xscale=1,yscale=0.33]
\draw [<->] (0,7) -- (0,0) -- (7,0);
\node [left] at (0,7) {$SR$};
\node [below right] at (7,0) {$I$};
%\draw [dashed, gray] (0,5) -- (7,5);
%\node [left] at (0,5) {\$5};
\draw [dashed, gray] (0,6) -- (7,6);
\node [left] at (0,6) {\$6};
%\draw [domain=0:13] plot (\x, {0.7999999*\x});
\draw [domain=0:7] plot (\x, {0.99*\x});
%\node [left] at (0,0) {0};
%\draw [dashed, lightgray] (5.05, 0) -- (5.05, 11);
%\node [below] at (5.05,0) {\$5.05};
\draw [dashed, lightgray] (6.06, 0) -- (6.06, 7);
\node [below] at (6.06,0) {\$6.06};
\end{tikzpicture}
\caption{Comparison of Impact as a Function of Security Risk for Design \#1}
\label{fig:SRvI}
\end{figure}
While both the values of the costs and security risk equations produce a dollar-unit metric, they are not immediately combinable as some costs are representative of life-time costs while others incorporate more immediate costs of a design. For this reason, a pairing of security risk and cost are used to determine the favorability of a given design in comparison of other design choices. It is worth noting that in the previous example, there was an assumption that the $c_a$ would be uniform no matter the system being attacked. To further add to the potential complexity of these calculations, one could take into account that there should be different values of $c_a$ depending on the type of security measures implemented into a given design; influencing the probability of an attack occuring (i.e. $p_a$). In other words, more complex security features may require much more costs to defeat those features. Now that a simple example has been shown, we now expand the considerations that are made for this simplistic example. The following section will examine further expansion of `estimation metric' considerations, showing how the calculation of comparable metrics can become more involved and complicated.
\section{Examining Attack and Defense with Detail}
\label{sec:attackDefense}
%The purpose of this section is to take the framework, along with the verification and selection process, and examine the stages of encryption and authentication and how including attack and defense considerations changes the landscape.
Depending on the point of view of the individual examining a system, the risk and cost analysis will yield different results to different individuals based on produced incentive and attractiveness of obtainable information in comparison to the defenses that must be circumvented and the related cost. This means that when accounting for the attacker and defender views, there will be further levels of considerations and some that can be wholly arbitrary. Certain aspects may be similar, such as the number of steps an individual must go through to interact with a system (e.g. subnets). While these do exist, the area of greater interest are the aspects that differ between the two points of view. To begin this process one needs to detail out any and all statistical information about attacks on various systems, how the hardware and software nuances affect the overall ranking of different methodologies, and most difficult, one must determine a method for applying the arbitrary qualitative sense of worth for specific data to one's equations in a meaningful way. Otherwise, the generated estimation metric becomes unintelligible.
%\begin{lstlisting}[caption={User-defined Higher Level Security Requirement},label={lst:AADLUserDefineHigh}]
%abstract implementation sysreq.wireless_sensor_i
% subcomponents
% serv_ADConv: abstract sysserv.ADConv_i {
% servatrb::dynamicRange => 0..5 V;
% secatrb::integrity::atkImpact => 300;
% };
% serv_wrlsTrans: abstract sysserv.wrlsTrans_i {
% servatrb::distance => 100 m;
% secatrb::authentication::atkValue => 600;
% secatrb::authentication::atkImpact => 400;
% secatrb::authorization::atkImpact => 1200;
% secatrb::dataleakage::atkImpact: => 800;
% secatrb::dataleakage::atkValue: => 800;
% };
% fnc_data: abstract security_props.data_i {
% dataatrb::data_class => Sensor;
% secatrb::atkImpact => 800;
% properties
% secatrb::hasProtection => false;
% secatrb::AuthGroup => Employees;
%end sysreq.wireless_sensor_i;
%\end{lstlisting}
One can further complicate these calculations by including a difference in examining the costs from the standpoint of both the defender of some sensitive information and the attacker trying to acquire any and all `useful' data.
%One example of this are the `atkImpact' and `atkValue' fields in Listing~\ref{lst:AADLUserDefineHigh}. These represent potential metrics of importance for different elements, attack vectors(e.g. Figure~\ref{fig:attackTree}), or worth of data that can be obtained by attacking a given system. As a defender, one can work towards detailing a system's defenses to have a better interpretation of the attack landscape.
For example: one can create a `correlation matrix' of the enacted defenses and the affect of failure of one defense on the other existing defense (0-1 scaled as well), as this would allow for greater detail of the system from the point of the defender. Additionally, this same information would be of great importance to a would-be attacker.
%\subsection{Expansion of Details}
While the examinations proposed here are specific to the system under examination, considerations can extend to a far larger scope of concern. One can begin to account for the worth of data to external parties (e.g. any other individual that is not the attacker or defender; black market or customers).
%Then one must account for the arbitrary worth of that information to another individual, which could also vary greatly.
For example: an encrypted file that a computer novice comes across will have little worth to them, but to an individual whom has experience this may pass some calculated risk threshold. Additionally, a person could act as an `information broker' that may not have a `worth' value attached to data at this point, but in the future the worth associated with specific information could easily rise or fall depending on content and `liveliness'.
%\section{Additional Concerns}
\section{Expanding Considerations}
\label{sec:additionalConcerns}
Despite the efforts to simplify the embedded system security modeling problem, there are a plethora of additional variables that can be taken into consideration depending on the environment and user, or company, at risk. When calculating the estimated cost metrics used for making larger system design decisions, one ideally accounts for as many aspects of the system as possible. When dealing with the constrained domain of embedded systems, this is doubly so. Physical costs of the system carry a heavy weight when making choices about design implementations. Power costs, heat generated, run time, area requirement, etc. are all aspects that must be considered. When incorporating software, the assumption of this paper is that software has been designed correctly with no error or malicious behavior embedded within. To that case, this means that when comparing software implementations, the main considerations of the developer are run time, bits used (in the case of encryption and authentication), and power consumed (total system run time on a single charge). To further add the weight of additional design considerations to our proposed verification and selection process, one can account for any additional features as a result of design decisions.
%This would mean that the inverted requirements weight would need to be further tailored to account for additional features that a design receives due to the decisions made by a development team.
In this manner a design team can further tweak the generation of certain design solutions based on the requirements met and additional features produced as result.
There can also be additional considerations that come about due to how probabilities, risk, and weights are chosen to be interpreted. Should an individual choose to incorporate features such as exposure or concentrate on sensitivity and security level of individual data, then one could attempt to produce a `probability rating' that represents these concerns.
%, as shown in Equation~\ref{equ:probRating}.
%
%\begin{multline} \label{equ:probRating}
% `Probability Rating' = weight_{sensitivity of data} * `known vulns metric' \\
%+ weight_{exposure} * Security Level Metric_{data?}
%\end{multline}
%
Complications of this examination range from simple questions, such as how does one measure exposure of an element, to more complicated concerns, such as how does one rank known vulnerabilities or how can one be aware of all vulnerabilities within a system.
%As such, this is perhaps not the best equation for producing an overall system metric, but may play a role in a more bounded examination.
% Explination of User Risk Type
`User risk type' (URT) comes from a business minded examination of risk~\cite{Mukhopadhyay2013} where users can be averse to risk, risk seeking, or risk neutral. The purpose of this variable would be as a method of representing different users of secure systems that exist within the real-world (e.g. users that are over protective and
users that treat security with a laissez-faire attitude). This would allow for the risk value produced to account for a degree of user tolerance. For example: the expected risk of two designs may be calculated to be \$10,000 (1\% of \$1,000,000 or 10\% of \$100,000) but the type of user expected to interact with the designed system leads to a preference for one design over an other. Thus, a developer must now also account for the behavior of the people using the system and incorporate how the user risk type affects the potential losses incurred. Risk tolerant users would not only allow for riskier designs but would also be willing to tolerate low probabilities of high loss. Likewise, risk averse users would lead to development of less risky designs that also limit the maximum potential loss.
%The User Risk Type should produce a lower value if a user is risk averse and a larger value if a given user is risk seeking. In this way, User Risk Type should `negatively' affect the cost of the system as more risk prone individuals become the user.
%Another method of examination would be to focus on the total `wealth' of a system over time. In this case one's focus would be more towards the future and use of a given system within an organization.~\cite{Ferrante2013}
%
%\begin{multline} \label{equ:wealthFunction}
% `Wealth Function' = Probability Nothing Happens * Initial Wealth \\
%+ Probability Event Happens * (Initial Wealth - Losses)
%\end{multline}
%
%`Initial Wealth' can be calculated from the design model generated, while the probability of events occurring will require more statistical harvesting of information relating to known attacks or event occurring within a given system. While advantageous from a business standpoint, this equation is also tailored to a specific point of view for examination of a given embedded systems security model.
%Expanding on this concept of point-of-view, then next section moves to examining the generated secure embedded system models through the eyes of both the defender and the attack as well as considerations that arise.
While the considerations, up to this point, have mainly been focused on how to define risk, how to interpret qualitative values of security into quantitative ones, and even how the examination of a system changes when incorporating attacker and defender points of view, there are still larger problems that must be addressed.
One of the largest problems facing security verification and validation techniques is the requirement for a `security expert' to formulate `libraries' of information and other data that will be needed for greater formalization of calculated values. Unfortunately these `libraries' are almost entirely arbitrary with respect to the knowledge and experience of a given security expert. This allows a great deal of room for inconsistencies due to human input. While Ferrante et.~al.~developed a method for locating inconsistencies within their work, there is no standardized methodology for determining inconsistencies in general. this means that these values can not be calculated in a deterministic manner and require a more `relatively deterministic' touch.
%An other concern is that current attempts at developing security metrics produce unit-less metrics; due to their arbitrary nature. For example: in the work by Ferrante et.~al.~they are able to produce a metric value, but the interpretation of that value requires an explanation by the researchers instead of allowing for a glance-interpretation of the data. This complicates manners of comparison and is a known problem when trying to develop new benchmarks and standards for comparison. This paper proposed a standard of maintaining a monetary-time value since most aspects of security incorporate time to some degree and it is relatively simple to associate a financial value to time spent.
The last, and perhaps largest concern, is that there are still aspects of the proposed process that are arbitrarily decided upon. This is an extremely difficult element to remove from the current examination of security framework verification and selection using an adversarial risk-based approach to embedded systems security modeling due to the need for qualitative input to a system that has yet to have standard quantitative input. While this work will continue to attempt to minimize the use of arbitrary values and rankings, it will take the effort of the entire security community to help move security measurements from a qualitative scale for a more quantitative one.
\section{Conclusion}
Having examined a new security modeling framework, coupled with the examination of its verification and selection process, we have introduced new meaningful security and risk-based metrics that take into consideration of worth to attacking and defending parties can complicate calculations due to a lack of knowledge and experience of how to measure and relate them. As the security community continues to develop and grow new techniques and methodologies, they must also rigorously standardize their method of examination to allow for better, more effective verification and selection of `best fit' embedded system security modeling solutions.
Future work include further development of the mapping process using a risk-based methodology, continue creating more deterministic means for describing security requirements and architectural component capabilities in a meaningful and comparable manner, and development towards an automated method for comparing generated solutions within a constrained and directed manner to arrive at best solutions for generated embedded systems security models.
%
% ---- Bibliography ----
%
\begin{thebibliography}{5}
\bibitem {Ferrante2013}
Ferrante, A., Milosevic, J., Janju\u{s}evi\`{c}, M.:
A Security-enhanced Design Methodology for Embedded Systems,
International Conference on Security and Cryptography (SECRYPT) (2013)
\bibitem {Mukhopadhyay2013}
Mukhopadhyay, A., Chatterjee, S., Saha, D., Mahanti, A., Sadhukhan, S.k.:
Cyber-risk decision models: To insure IT or not?,
Decision Support Systems, Volume 56 pages 11--26 (2013)
\bibitem {Vincentelli2007}
Sangiovanni-Vincentelli, A.:
Quo Vadis, SLD? Reasoning About the Trends and Challenges of System Level Design,
Proceedings of the IEEE (March 2007)
\bibitem {pal2014will}
Pal, R., Golubchik, L., Psounis, K., and Hui, P.:
Will Cyber-Insurance Improve Network Security? A Market Analysis,
IEEE INFOCOM 2014-IEEE Conference on Computer Communications, Pages 235--243 (2014)
\bibitem {fauntleroy2015cyber}
Odell, L.A., Fauntleroy, J.C., and Wagner, R.R.:
Cyber Insurance - Managing Cyber Risk,
Institute for Defense Analysis (April 2015)
\bibitem {biener2015insurability}
Biener, C., Eling, M., and Wirfs, J.H.:
Insurability of Cyber Risk: An Empirical Analysis,
The Geneva Papers on Risk and Insurance Issues and Practice,
Volume 40 Issue 1 pages 131--158 (January 2015)
\bibitem {armando2004satmc}
Armando, A. and Compagna, L.:
SATMC: A SAT-Based Model Checker for Security Protocols,
Logics in Artificial Intelligence, Volume 3229 pages 730--733 (2004)
\bibitem {brucker2012securebpmn}
Brucker, A.D., Hang, I., L\"{u}ckemeyer, G., and Ruparel, R.:
SecureBPMN: Modeling and Enforcing Access Control Requirements in Business Processes,
Proceedings of the 17th ACM symposium on Access Control Models and Technologies, pages 123-126 (2012)
\bibitem {gonzalez2012quantitative}
Gonzalez, N., Miers, C., Red\'{i}golo, F., Simpl\'{i}cio, M., Carvalho, T., N\"{a}slund, M., Pourzandi, M.:
A quantitative analysis of current security concerns and solutions for cloud computing,
Journal of Cloud Computing: Advances, System and Applications Advances, Systems and Applications (2012)
\bibitem {mukherjee2013attributed}
Mukherjee, D.:
Attributed Metagraph Modelling to Design Business Process Security Management,
Journal: International Letters of Social and Humanistic Sciences,
Issue 06 pages 41--48 (2013)
\bibitem {marotta2015survey}
Marotta, A., Martinelli, S.N., Yautsiukhin, A.:
A Survery on Cyber-Insurance,
Technical Report (November 2015)
%\bibitem {SysML-Sec}
%SysML-Sec,
%\url{http://sysml-sec.telecom-paristech.fr/}
%
%\bibitem {jurjens2005secure}
%J{\"u}rjens, J.:
%Secure systems development with UML,
%Springer Science \& Business Media (2005)
%
%\bibitem {jurjens2002umlsec}
%J{\"u}rjens, J.:
%UMLsec: Extending UML for secure systems development,
%UML 2002—The Unified Modeling Language, Springer Publishing, pages 412--425 (2002)
%
%\bibitem {SysML}
%SysML,
%\url{http://sysml.org/}
%
%\bibitem {AADLSite}
%AADL,
%\url{http://www.aadl.info/aadl/currentsite/}
%
\bibitem {AADLV2Overview}
Feiler, P.:
SAE AADL V2: An Overview.
Carnegie Mellon University (2010)
%
%\bibitem {AADLTools}
%AADL Tools,
%\url{https://wiki.sei.cmu.edu/aadl/index.php/AADL\_tools}
%
%\bibitem {Osate2}
%Osate 2,
%\url{https://wiki.sei.cmu.edu/aadl/index.php/Osate\_2}
%
%\bibitem {Osate2Examples}
%Osate 2 Example Repository,
%\url{https://github.com/osate/examples}
%
%\bibitem {UserDaysMay2016}
%User Days - May 2016,
%\url{https://github.com/saeaadl/userdays/tree/master/UserDays/May2016}
%
%\bibitem {AADLResolute}
%Resolute Website,
%\url{http://loonwerks.com/tools/resolute.html}
%
%\bibitem {RDALOverview}
%Blouin, D.:
%AADL Requirements Annex Review,
%\url{http://www.aadl.info/aadl/downloads/committee/feb2013/presentations/aadl\_standards\_requirements\_annex\_review\_06022013.pdf}
%
%\bibitem {gacek2014resolute}
%Gacek, A., Backes, J., Cofer, D., Slind, K., Whalen, M.:
%Resolute: An assurance case language for architecture models,
%ACM SIGAda Ada Letters, Volume 34 Number 3, pages 19--28 (2014)
%
%\bibitem {EMV1}
%Feiler, P.:
%SAE AADL Error Model Annex: An Overview,
%\url{https://wiki.sei.cmu.edu/aadl/images/1/13/ErrorModelOverview-Sept222011-phf.pdf}
%
%\bibitem {EMV2}
%Feiler, P.:
%SAE AADL Error Model Annex: Discussion Items,
%\url{https://wiki.sei.cmu.edu/aadl/images/1/13/ErrorModelOverview-Sept222011-phf.pdf}
%
%\bibitem {BLESS2013}
%Larson, B.R., Chalin, P., Hatcliff, J.:
%BLESS: Formal Specification and Verification of Behaviors for Embedded Systems with Software,
%\url{https://ti.arc.nasa.gov/m/events/nfm2013/pubs/BLESS.pdf}
%
%\bibitem {AADLSecAnnex}
%Delange, J., Feiler, P., Klieber, W., Nam, M., Seibel, J.:
%AADL Security Annex,
%\url{https://github.com/saeaadl/userdays/blob/master/UserDays/May2016/security-annex-May2016.pdf}
%
%\bibitem {AADLSecAnalysis}
%Delange, J., Nam, M., Seibel, J.:
%AADL Security Analysis Tools,
%\url{https://github.com/saeaadl/userdays/blob/master/UserDays/May2016/security-analysis-May2016.pdf}
%
%\bibitem {ellison2015extending}
%Ellison, R., Householder, A., Hudak, J., Kazman, R., Woody, C.:
%Extending AADL for Security Design Assurance of Cyber-Physical Systems,
%Software Engineering Institute, CMU/SEI-2015-TR-014 (2015)
%
%\bibitem {jeepHack}
%Drozhzhin, A.:
%Black Hat USA 2015: The full story of how that Jeep was hacked,
%\url{https://blog.kaspersky.com/blackhat-jeep -cherokee-hack-explained/9493/}
%
%\bibitem {planeHack}
%Zetter, K.:
%Feds say that banned researcher commandeered a plane,
%\url{http://www.wired.com/2015/05/feds-say-banned-researcher-commandeered-plane/}
%
%\bibitem {superFish}
%Hope, P.:
%Superfish adware weakens security and injects ads on some Lenovo laptops,
%\url{http://www.techrepublic.com/article/superfish-adware-weakens-security-and-injects-ads-on-some-lenovo-laptops/}
%
%\bibitem {lenovoWPBT}
%Sanders, J.:
%Windows and UEFI anti-theft mechanism makes systems less secure,
%\url{http://www.techrepublic.com/article/windows-and -uefi-anti-theft-mechanism-makes-systems-less-secure/}
%
%\bibitem {govHack}
%Olorunnipa, T.:
%Breach of Employee Data Wider Than Initial Report, U.S. Says,
%\url{http://www.bloomberg.com/politics/articles/2015-06-12/white-house-says-personnel-records-possibly-breached-twice}
%
%\bibitem {stageFright}
%Vaughan-Nicholas, S.J.:
%Stagefright: Just how scary is it for Android users?,
%\url{http://www.zdnet.com/article/stagefright-just-how-scary-is-it-for-android-users/}
%
%\bibitem {stageFright2}
%Whittaker, Z.:
%Stagefright is back, and affecting millions of Android devices,
%\url{http://www.zdnet.com/article/new-stagefright-2-0-flaws-affect-millions-of-android-devices/}
%
%\bibitem {androidUpdates}
%Tofel, K.:
%HTC says monthly Android security updates are ``unrealistic'',
%\url{http://www.zdnet.com/article/htc-says-monthly-stagefright-android-security-updates-are-unrealistic/}
%
%\bibitem {androidMarshmallow}
%Jack Wallen, J.:
%The woes of Android updates, and how to fix the process,
%\url{http://www.techrepublic.com/article/the-woes-of-android-updates-and-how-to-fix-the-process/}
%
%\bibitem {googleAndroid}
%Sanders, J.:
%Google finally doubles down on security with monthly Android updates,
%\url{http://www.techrepublic.com/article/google-and-some -android-phone-vendors-introduce-welcome-changes-to-security-update-process/}
%
%\bibitem {aaraj2008analysis}
%Aaraj, N., Raghunathan, A., Jha, N.K.:
%Analysis and design of a hardware/software trusted platform module for embedded systems,
%ACM Transactions on Embedded Computing Systems (TECS), Volume 8 Number 1, page 8 (2008)
%
%\bibitem {denning1996location}
%Denning, D.E., MacDoran, P.F.:
%Location-based authentication: Grounding cyberspace for better security,
%Computer Fraud \& Security, Volume 1996 Number 2, pages 12--16 (1996)
%
%\bibitem {saito2015case}
%Saito, M., Hazeyama, A., Yoshioka, N., Kobashi, T., Washizaki, H., Kaiya, H., Ohkubo, T.:
%A case-based management system for secure software development using software security knowledge,
%Procedia Computer Science, Volume 60, pages 1092--1100 (2015)
%
%\bibitem {denning2015toward}
%Denning, D.E.:
%Toward more secure software,
%Communications of the ACM, Volume 8 Number 4, pages 24--26 (2015)
%
%\bibitem {nguyen2015model}
%Nguyen, P.:
%Model-Driven Security With Modularity and Reusability For Engineering Secure Software Systems,
%University of Luxembourg (2015)
%
%\bibitem {ravi2004security}
%Ravi, S., Raghunathan, A., Kocher, P., Hattangady, S.:
%Security in embedded systems: Design challenges,
%ACM Transactions on Embedded Computing Systems (TECS), Volume 3 Number 3, pages 461--491 (2004)
%
%\bibitem {gokhale2008model}
%Gokhale, A., Balasubramanian, K., Krishna, A.S., Balasubramanian, J., Edwards, G., Deng, G., Turkay, E., Parsons, J., Schmidt, D.C.:
%Model driven middleware: A new paradigm for developing distributed real-time and embedded systems,
%Science of Computer programming, Volume 73 Number 1, pages 39--58 (2008)
%
%\bibitem {perez2006vtpm}
%Perez, R., Sailer, R., van Doorn, L., and others:
%vTPM: virtualizing the trusted platform module,
%Proc. 15th Conf. on USENIX Security Symposium, pages 305--320
%
%\bibitem {yan2015novel}
%Yan, W., Tehranipoor, F., Chandy, J.A.:
%A Novel Way to Authenticate Untrusted Integrated Circuits,
%Proceedings of the IEEE/ACM International Conference on Computer-Aided Design, pages 132--138 (2015)
%
%\bibitem {tehranipoor2015dram}
%Tehranipoor, F., Karimina, N., Xiao, K., Chandy, J.:
%DRAM based intrinsic physical unclonable functions for system level security,
%Proceedings of the 25th edition on Great Lakes Symposium on VLSI, pages 15--20 (2015)
%
%\bibitem {CommonCriteria}
%Common Criteria for Information Technology Security Evaluation,
%ISO/IEC, Number ISO/IEC 15408, July 2015
%
%\bibitem {benzel2005design}
%Benzel, T.V., Irvine, C.E., Levin, T.E., Bhaskara, G., Nguyen, T.D., Clark, P.C.:
%Design principles for security (2005)
%
%\bibitem {lin2013security}
%Lin, C., Zhu, Q., Phung, C., Sangiovanni-Vincentelli, A.:
%Security-aware mapping for CAN-based real-time distributed automotive systems,
%Computer-Aided Design (ICCAD), 2013 IEEE/ACM International Conference on, pages 115--121 (2013)
%
%\bibitem {markose2008systematic}
%Markose, S., Liu, X., McMillin, B.:
%A systematic framework for structured object-oriented security requirements analysis in embedded systems,
% IEEE/IFIP International Conference on Embedded and Ubiquitous Computing, 2008. EUC'08, Volume 1, pages 75--81 (2008)
%
%\bibitem {yu1997towards}
%Yu, E.S.:
%Towards modelling and reasoning support for early-phase requirements engineering,
%Proceedings of the Third IEEE International Symposium on Requirements Engineering, pages 226--235 (1997)
%
%\bibitem {massacci2010security}
%Massacci, F., Mylopoulos, J., Zannone, N.:
%Security requirements engineering: the SI* modeling language and the secure tropos methodology,
%Advances in Intelligent Information Systems, pages 147--174 (2010)
%
%\bibitem {sangiovanni2007quo}
%Sangiovanni-Vincentelli, A.:
%Quo vadis, SLD? Reasoning about the trends and challenges of system level design,
%Proceedings of the IEEE, Volume 95 Number 3, pages 467--506 (2007)
%
%\bibitem {ALISA2016}
%Delange, J., Feiler, P., Neil, E.:
%Incremental Life Cycle Assurance of Safety-Critical Systems,
%8th European Congress on Embedded Real Time Software and Systems (ERTS 2016)
\end{thebibliography}
\end{document}
You can’t perform that action at this time.