Results.tex

\chapter{Results}

There are results for each of the research questions, and some combined results.

\section{What do students think a proof is?}
Some students, when asked what they think a proof is, will report that they think it is a list of true mathematically formulated statements, demonstrating the truth of a mathematically formulated statement. Some students report that a proof has a goal, a statement to be proved true.
Some students know that the identified goal does not have to be known to be true in advance of the first proof.

Sometimes, though, students have the idea that the proof is an exhibit of their ability to connect known facts, including the goal as a known fact.



Some students, for example some taking philosophy, understand a proof more generally as not having a requirement for a mathematical formulation. Some of these have expressed dislike of such less precisely articulated statements.

Some students, when prompted, will acknowledge that warrants for these statements are required. Axioms and agreed facts do not require warrants. 
Some students, but not all, recognize that premises do not require warrants.
Some students, but not all, recognize that suppositions, as premises, do not require warrants.
Some students, but not all, recognize that cases, as suppositions, do not require warrants.
Some students, but not all, know that progress from one statement to the next, a transformation of a statement, requires a warrant.

Some of the optional syntactic ornamentation of a proof, such as literal text "Proof:", and "QED" or $\qed$, are used by some students as proxies for the proof. As in the research by Harel and Sowder\cite{harel1998students}, which they describe as "ritual proof", we find in our research that some students claim to recognize a proof when they see these artifacts, and claim they have not seen a proof when they do not see these artifacts.

Some students are aware that proof, as encountered in class, ought to be a convincing argument.
These students feel that something is wrong when they are not convinced by the proof technique they have learned to execute in a procedural fashion.

Some students know that proof is convincing others, and also ascertaining for oneself. Of these, some find that proof is convincing for some facts they regard as mathematical, yet do not think proof is applicable to programs as large as those with which they plan to be involved.
Some of these students have not yet acquired the perspective that proving theorems about the number of instruction executions, and/or memory locations needed are both numerical and also applicable to and relevant for software development.


\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/whatThemes}
\caption{Conceptualizations found for what a proof is}
\label{fig:whatThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline List of Known Facts & List of Warranted Facts & Warrants\\ 
\hline List of Warranted Facts & Means of Discovery & Tool rather than\\  & & demonstration to teachers\\ \hline
\end{tabular}
 \end{table}

\newpage
\section{How do students approach understanding a proof?}
Those students who felt they understood some proofs approached them by checking whether they felt each line of a proof was true.
Some of these mentioned that a statement should be warranted by previous statements.

\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/howThemes}
\caption{Conceptualizations of how to comprehend a proof}
\label{fig:howThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline List of Known Facts & List of Warranted Facts & Warrants\\ 
\hline List of Warranted Facts & Means of Discovery & Tool rather than\\  & & demonstration to teachers\\ \hline
\end{tabular}
 \end{table}

\newpage
\section{What do students think  a proof is for?}
Some students think that proofs are not applicable to what they do.

Some students claim that they never produce proofs unless assigned to do so in class.

Let us call the statement to be proved, the target, so as to more clearly articulate the variety of student thinking, by escaping the connotations of "statement to be proved".

Perhaps not surprisingly in light of the teaching of proof, some students think the purpose of proof is to demonstrate that they can construct a sequence of statements that connects the truth of the premises to the truth of the target. Some of these students regard the truth of the target to be known beforehand. As the purpose of the proof is to exhibit their ability to produce an argument, it is not surprising that students say they never construct proofs unless they are assigned to do so. It is not surprising in this context, that students opt for a procedural approach, learning the parts, for example, of a proof by induction, learning to provide a proof for a base case, and learning to take a premise as a given and a conclusion of an implication to be proved. Some students do not understand why this procedure constructs a proof. Some express an unease -- they wish for the proof procedure to be convincing, to themselves. They are glad when they learn why the procedure does produce a convincing argument.

Some students recognize proof being used in class, for example in algorithms class and in introduction to the theory of computing.




Some students felt that proof was for finding out that a mathematical expression was true, or false. Some students knew that some statements could be proved undecidable.

\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/whyThemes}
\caption{Conceptualizations about why to study proofs}
\label{fig:whyThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline List of Known Facts & List of Warranted Facts & Warrants\\ 
\hline List of Warranted Facts & Means of Discovery & Tool rather than\\  & & demonstration to teachers\\ \hline
\end{tabular}
 \end{table}

\newpage
\section{What do students use proof for, when not assigned?}
Some students claim they never use proofs when not assigned.

It is not the case that any student, even when prompted, said they chose to carry out a proof without being directed to do so.
This could easily be due to a misunderstanding of the definition of proof.

\newpage
\section{Do students exhibit any consequence of inability in proof?}
Some students said that they knew how to craft recursive procedures, and enjoyed doing so when assigned problems designated as suitable for recursive implementations.
Some students said they did not employ recursive procedures in situations without a designation that recursive procedures were appropriate. They claimed not to be able to tell when recursive procedures were applicable.

\newpage
\section{What kind of structure do students notice in proofs?}
Some students think proofs are lists of statements without hierarchical structure.
Some students have asked what lemma means.
Some students knew that lemmas were built for use in larger proofs.
Some students were interested to hear about Dr. Lamport's structure in proofs.

\newpage
\section{What do students think it takes to make an argument valid?}
Some students, when prompted about rules of inference, felt that when all statement transformations were warranted, an argument was valid.
Some students stated that, when the target of the proof was true, the proof was valid, converse error.





We organize our overview of results beginning from an ideal Hilbert-axiomatic style of proof approach, and moving through approximations as they become greater departures from it.
\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/valid}
\caption{Conceptualizations about validity of proofs}
\label{fig:validityThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a (valid) proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline No Warrants & Some Warrants & Warrants\\ 
\hline Some Appropriate Warrants & Fully Warranted & thoroughness\\    \hline
\end{tabular}
 \end{table}

\paragraph{Definition based reasoning}
Some students, and some teaching assistants in their teaching, are not organizing the approach to proof around definitions.
Instead some students and some teaching assistants are focusing on an intuitive approach, involving examples.
Some students use examples to infer definitions.
Some students use single examples as proof.

Some students are not aware that proofs are illustrated with facts of, for the purposes of the class, less significance than the proof techniques. 
Some students are not aware of the relevance of proof to their intended career.
These students do not see any point to learning more than a procedural approach to the proof material, as they believe it to be of no lasting significance to them.

\paragraph{Generalization and transformation based reasoning}
Some students, and some instructors, do not emphasize that a single presentation can be seen as a representative of a group. For example, Mathematical  Association of America\cite{} publishes a proof of the Pythagorean Theorem that uses rectangles to illustrate that, when they are square, the Pythagorean Theorem is being shown to be true, though of course, the rectangles need not always be square. The proof, having been established, does not rely upon the rectangles remaining in a square condition.

 





\section{Combined Description}

There are a couple of ways students work with exercises in proof, that are incomplete.

Some students reason well with concrete entities, yet are confused with abstractions. These students are not appreciating the value of careful definitions, because they do not use them as tools, or the basis for reasoning. They are more comfortable with examples, because they are operating in a concrete world.

Some students do not connect the world of concrete objects with the abstract, symbolic representation, but are making use of symbols. Some operations transforming symbolic expressions are performed correctly, but not all. The lack of understanding of the symbols combined with a procedural approach to producing a proof artifact, leaves these students personally unconvinced, and unmotivated to make use of proof when it would be helpful to them.

Some students do understand application of facts, axioms and rules of inference, and are at home with careful definitions and symbolic concision. Some of these students also study math.


\newpage
\section{Diagram of Conceptualizations}

\begin{figure}[h]
\centering
\includegraphics[width=0.95\linewidth]{./themes}
\caption{Themes from interview data}
\label{fig:themes}
\end{figure}
\newpage
\section{Outcome Space}
The outcomes were not arranged in a single progression. Rather, there were several means, listed below, by which students were able to construct the proof artifact required by the class. The students did not always find the artifact convincing.

\begin{enumerate}
\item Concrete to Abstract -- generalize the argument, then the entities
\item Hilbert-style axiomatic/definitional proof
\item Abstract operations  -- symbols rather than entities, structure of argument
\end{enumerate}

The concrete to abstract path enabled students to reason with specific cases whose logic made sense to them, then make the step that the logical process itself was an entity that could be reused. The idea that other concrete entities could bear the same relationships, and be subject to the same reasoning constituted a step. The idea that analogies were being made, and that generalization was possible was another step.

The reasoning by axioms and rules of inference path was known to some students. These students mentioned their appreciation of math, and in some cases their discomfort with philosophy, in connection with symbolization and application of rules of inference.

One path was operation at the level of symbols, using procedures. This path is distinguished from that involving definitions, because some students, using definitions, were clear about appropriate operations to transform symbolic expressions, but students also sometimes were unsure about denotation and about appropriate operations.

\section{Critical Factors}
On the path from concrete to structured proofs, called herein "generalize the argument, then the entities", one critical factor is that an argument about one set of concrete entities can be used on another set, having analogous relationships.

Another critical factor is that, when an argument can be reused, sets of entities that stand in analogous relationships, the relationship can be generalized. When the relationship is generalized, the entities standing in that relationship can be given symbols.

On the path that starts with symbols, students have not generalized from concrete to abstract entities, rather, they have entered the fray at the level of abstraction of symbols. Thus, a critical factor is to understand the operations appropriate to the symbols, which imbues application of the rules of inference with significance. Another critical factor is that these symbols can represent entities of interest.





\section{Earlier Paper Material}

\subsection{Categories of Experience of Entering Students}
Undergraduate students beginning study of the computing disciplines present
various degrees of preparedness\cite{smith2013categorizing}. Some had no experience, some had had
informal experience, and some had had formal classes. The formal classes
extended from using applications to building applications. Informal experience
ranged from editing configuration files, such as background colors, to full time
jobs extended over multiple summers.
After publishing this paper, we encountered more related information. For
example, consistent with the work of Almstrum\cite{almstrum1996investigating}, we found that, about implications,
some students, who do know that any statement must and can, be
either true or false, thought implications must be true. Some interview participants
enjoyed a modified Moore method\cite{cohen1982modified} geometry class in middle school,
and relished opportunities to create proofs (not yet published). Other students
were not so well prepared.
\subsection{Representation/Symbolization in Pumping Lemmas}
We found that some students may lack facility in notation. For example, in the
application of the pumping lemma, students are expected to understand the
role of $i$, in the context that a string $s$, having component substrings $x$, $y$ and $z$,
can be used to generate other strings, of the form $xy^iz$, where $i$ gives the number
of copies of the substring $y$. Moreover, students are expected to understand that
the subdivision of a string of length $p$, $\sigma_1^a\sigma_2^{p-a}$, where $a \in \{0,1,\ldots,p\}$
uses $a$ as a parameter, a free variable, not one necessarily bound to a single instance
of a natural number, but a representation of a domain. An excerpt of the errors found on tests is shown in Table . Trigueros et
al. \cite[ p. 3]{jacobs2008developing} have observed that ``students are often unclear about the different
ways letters are used in mathematics''.  We have seen this
lack of understanding in a situation in which it was proposed as evidence that
a single example, namely $\sigma_1^a\sigma_2^{p-a}$, formed a proof for a universally quantified
statement.
Some of our results were consistent with the framework described by Harel
and Sowder in 1998\cite{harel1998students}. We found students holding conceptualizations that
Harel and Sowder's 1998 model calls symbolization. Harel and Sowder have
identified another category of conceptualization, that correctly applied transformation
and axiomatic arguments. Some students expressed enthusiasm for
the power that inheres to building arguments with carefully specified component
ideas, in particular how the absence of ambiguity permitted arguments to
extend to great length while remaining valid. Not all of the students had developed
axiomatic conceptualizations of proof. About definitions, we collected
preliminary data on students' conceptualizations of definitions used in proofs.
Some students thought definitions were boring. Some students thought that
they could infer definitions from a few examples. Concerning executive function,
we found that some students do not state the premises clearly, and some
students did not keep track of their goal. About rules of inference, we found
Figure 5.3.1: Some categories / conceptualizations found among students of
introduction to the theory of computing, and published at FIE.
that some students apply invalid approaches to inference.

\subsection{Abstract Model for Proof by Mathematical Induction and Recursion}
Far from finding agreement that (a) theorems are true as a consequence of
the definitions and the premise, and that (b) proofs serve to show how the
consequence is demonstrated from the premise, axioms and application of
rules of inference, instead we found a variety of notions about proof, including
the well-known procedural interpretation \cite{tall2008transition,weber2004traditional,tall2001symbols}, and the well-known empirical
misconception \cite{harel1998students}. The conceptualization that definitions are not necessarily
of interest compared with the procedures seemed different in kind from the
concept image / concept definition discoveries of R\"osken et al. \cite{rosken2007integrating}.
Interviews with students revealed that some students saw generation of a proof
by mathematic induction as a procedure to be followed, in which they should
produce a base case, and prove it, and should produce an induction step, and
prove that. This was consistent with Weber [?, p. 4-426] who has stated ``in
the studies that I conducted, it was more often the case that undergraduates
applied procedures that were not meaningful to them.'' He went on to give a
quotation from a participant [?, p. 4-426] ``And I prove something and I look at
it, and I thought, well, you know, it's been proved, but I still don't know that I
even agree with it [laughs]. I'm not convinced by my own proof!'' Some of the
students interviewed did not know why this procedure generated a convincing
argument. Polya[?] has written a problem involving all girls being blue-eyed;
a similar problem appears in Sipse \cite{sipser2012introduction} about all horses being the same color.
The purpose of this exercise is to make students aware that the truth of the
inductive step must apply when the base case appears as the premise. In some
cases, this point was not clear to the students.
Students' conceptualizations of proof by mathematical induction can support
their choice to apply recursive algorithms. One student reported success at
both mathematical induction and recursive algorithm application without ever
noticing any connection. This student opined that having learned recursion
with figures, and proof by mathematical induction without figures, that no
occasion for the information to spontaneously connect occurred. Students reporting
ability to implement assigned problems recursively, but not the ability
to understand proof by mathematical induction also reported that ability to
write recursive programs did not result in recognition of when recursive solutions
might be applicable in general. Students reporting ability to implement
assigned problems recursively, and also the ability to prove using mathematical
induction also reported preferring to implement recursive solutions in
problems as they arose.
Our work on students' choices of algorithmic approaches was consistent with
work by other researchers in computer science education\cite{booth1997phenomenography} on conceptualizations
of algorithms. Our work served to unify that of mathematician educators
with computer science educators, by providing a plausible explanation why
the conceptualizations of recursive algorithms that were found, might exist.
\subsection{Proofs by Induction}
Table 5.3.3: The Outcome Space for Proofs by Induction
Category Description
1Following procedure The method is learned, without understanding
2Understands base case The idea that a base case is proved by an existence
proof, often with a specific example
3Understands implication The idea that an implication is proved by
assuming the premise is not used
4Does not understand connection
Sees the implication and proves it well, but
does not anchor the succession to a base
case
5Does understand the argument
Understands the argument
6Knows why recursion
works
Can tailor the argument to explain recursive
algorithms
7Appreciates data structures
supporting recursion
Can see the benefit to algorithm from recursive
data structure
\subsection{Pumping Lemmas}
TABLE III. CATEGORIES\\
understand inequality\\
formulate correctly\\
distinguish between particular and generic particular\\
correctly apply universal quantifier\\
recognize string as member of language set

\section{Discussion}
 \subsection{Importance}
Importance goes here, rather than in analysis\\

Programmers/developers who produce and/or verify software that is used in safety critical applications, such as medical equipment, self-driving cars, and defense-related equipment should be able to know that their software functions correctly.

Programmers/developers who produce and/or verify software that is expected to perform work, such as search, efficiently, should be able to know that their algorithms are efficient.

Computer science is the expected background preparation for people working in these careers.
Proof is the method that is used to ascertain, and to convince, that these goals have been achieved.
\subsection{Interpretation of Results}
As in mathematics, some students learn as procedure that which we would prefer that they understand.
Some procedural learning is insufficiently accompanied by an understanding as to which contexts to which it applies, and has become in some cases what Whitehead calls "inert knowledge".

\subsection{More Discussion}
\paragraph{Helping Students Discern Abstraction}
Recall that variation theory holds that students cannot discern a thing unless
contrast is provided. Pang has pointed out that [], for persons aware of only
one language, ``speaking'' and ``speaking their language'' are conflated. Only
when the existence of a second language is known, does the idea of speaking
become separated from the idea of speaking a specific language.
(Here is a specialization (Hofstadter), formation of a new conjunct (Valiant), see Besold 2015)
Abstraction is important in computer science, and is worthy of investigation.
Inquiry into students' conceptualizations of formalization using symbols, symbolization,
has shown similar results among students of mathematics and of
computer science [?, ?]. Student populations contain the conceptualization that
proofs ought to be expressed using symbols, and some proof attempts show
that not all students are able to formalize meaningfully. Mathematics and computer
science pedagogies differ on the recommended style of variable names in
symbolization. In mathematics, there is a preference for single letter variable
names, and in computer science it is recognized that longer variable names assist
readers in understanding. In mathematics the use of single variable names
is preferred because it is thought to contribute to cultivating students' ability
to learn abstraction. If, in computer science education, we apply variation
Table 5.3.5: The Outcome Space for Proofs by Induction\\
Category Description\\
 1 Following procedure The method is learned, without understanding\\
  2 Understands base case The idea that a base case is proved by an existence
proof, often with a specific example\\
3 Understands implication The idea that an implication is proved by
assuming the premise is not used\\
4 Does not understand connection
Sees the implication and proves it well, but
does not anchor the succession to a base
case\\
5 Does understand the argument
Understands the argument\\
6 Knows why recursion
works\\
Can tailor the argument to explain recursive
algorithms\\
7 Appreciates data structures
supporting recursion\\
Can see the benefit to algorithm from recursive
data structure\\
theory, we gain confidence in the idea that students may discern the process
of abstraction as we vary the names of the variables. We could imagine deriving
code from a requirement about a specific class, and using corresponding
variable names, and we could show the process of promoting the code into a
more general class in the inheritance hierarchy, changing the variable names to
correspond to the more general domain of objects. Thus we can borrow from
the approach used by mathematics education, but make it more explicit, taking
advantage of computer science's explicit treatment of inheritance hierarchies in
object oriented code. Seeking evidence of students' conception of abstraction,
we could examine overridden methods to see whether variable names in more
and less general implementations bear that relation to one another.
\paragraph{Helping Students Discern Abstraction}
\paragraph {Algebra}
In middle or high school algebra students became familiar with the use of letters in equations,
and solving equations which resulted in individual values, or no value, being attached to the letters.

Ideally the ability to understand expressions, to formulate pre- and post conditions would be acquired.
As we have seen that this occurs sometimes, but does not always occur,
there may be benefit to some students to review this idea.
We might choose to emphasize abstraction in this process.

\paragraph{Geometry}
In high school geometry, formal proofs of geometric properties are covered.
Students are exposed to a form for argument, and are given examples of use of rules of inference to perform logical deduction.
We have seen that sometimes this process is appreciated in enough generality to be recognized
as an example of argumentation.
We have seen as well, that some students found this process entirely specific to geometry,
doubting that it had broader application.

\paragraph{Seeing a Broader Context}

It may be that some students do not see a separation between the activity of formalization on the one hand,
and the application area of finding solutions to equations on the other.
It may be that some students do not see a separation between the activity of deducing using logic on the one hand, and the application area of learning geometric facts on the other.
It seems consistent with neglecting opportunities for abstraction, that these separations are not always seen,
Speciation, an idea which uses abstraction, providing a hierarchy of properties animals and plants might have, was not recognized early or universally. So, it is
not surprising that abstraction, which involves choice about which details to defer, and which to regard as significant, is not always obvious.
In the machine learning perspective, features can be learned.
What do I want to say, it takes some effort to recognize features?
There might be a way to formulate choice of features such that some better efficiency is gained by thinking of the features in that order vs. another order.
(Such as, we never have to think about some features for some parts of the tree.)
If we think about knowledge being organized in neural networks, such that abstraction has a physical manifestation, we can see that ideas between which there is little distance (by some measure, neurons, glia?) in the tree will more frequently elicit one another by linkages at the metabolic level.
At the neural level, modifications for efficiency are constantly taking place (Do we have this from Kandel and Squire?).
We might wish to exploit this in the way we teach, to exhibit the abstraction deliberately, to minimize the amount of neural connection remodeling that would occur in the process of providing an efficient neural connection remodeling that would occur in the process of providing an efficient neural representation.
Being able to learn by analogy testifies to the utility of having a neural representation that corresponds to abstraction.
Students who are working without hierarchical organization of concepts are at a disadvantage.
\paragraph{Mathematics tests in high school that involve proving}
What can we learn from students of computer science who excelled in reasoning to this level?


There are results for each of the research questions, and some combined results.

\section{What do students think a proof is?}
Some students, when asked what they think a proof is, will report that they think it is a list of true mathematically formulated statements, demonstrating the truth of a mathematically formulated statement. Some students report that a proof has a goal, a statement to be proved true.
Some students know that the identified goal does not have to be known to be true in advance of the first proof.

Sometimes, though, students have the idea that the proof is an exhibit of their ability to connect known facts, including the goal as a known fact.



Some students, for example some taking philosophy, understand a proof more generally as not having a requirement for a mathematical formulation. Some of these have expressed dislike of such less precisely articulated statements.

Some students, when prompted, will acknowledge that warrants for these statements are required. Axioms and agreed facts do not require warrants. 
Some students, but not all, recognize that premises do not require warrants.
Some students, but not all, recognize that suppositions, as premises, do not require warrants.
Some students, but not all, recognize that cases, as suppositions, do not require warrants.
Some students, but not all, know that progress from one statement to the next, a transformation of a statement, requires a warrant.

Some of the optional syntactic ornamentation of a proof, such as literal text "Proof:", and "QED" or $\qed$, are used by some students as proxies for the proof. As in the research by Harel and Sowder\cite{harel1998students}, which they describe as "ritual proof", we find in our research that some students claim to recognize a proof when they see these artifacts, and claim they have not seen a proof when they do not see these artifacts.

Some students are aware that proof, as encountered in class, ought to be a convincing argument.
These students feel that something is wrong when they are not convinced by the proof technique they have learned to execute in a procedural fashion.

Some students know that proof is convincing others, and also ascertaining for oneself. Of these, some find that proof is convincing for some facts they regard as mathematical, yet do not think proof is applicable to programs as large as those with which they plan to be involved.
Some of these students have not yet acquired the perspective that proving theorems about the number of instruction executions, and/or memory locations needed are both numerical and also applicable to and relevant for software development.


\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/whatThemes}
\caption{Conceptualizations found for what a proof is}
\label{fig:whatThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline List of Known Facts & List of Warranted Facts & Warrants\\ 
\hline List of Warranted Facts & Means of Discovery & Tool rather than\\  & & demonstration to teachers\\ \hline
\end{tabular}
 \end{table}

\newpage
\section{How do students approach understanding a proof?}
Those students who felt they understood some proofs approached them by checking whether they felt each line of a proof was true.
Some of these mentioned that a statement should be warranted by previous statements.

\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/howThemes}
\caption{Conceptualizations of how to comprehend a proof}
\label{fig:howThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline List of Known Facts & List of Warranted Facts & Warrants\\ 
\hline List of Warranted Facts & Means of Discovery & Tool rather than\\  & & demonstration to teachers\\ \hline
\end{tabular}
 \end{table}

\newpage
\section{What do students think  a proof is for?}
Some students think that proofs are not applicable to what they do.

Some students claim that they never produce proofs unless assigned to do so in class.

Let us call the statement to be proved, the target, so as to more clearly articulate the variety of student thinking, by escaping the connotations of "statement to be proved".

Perhaps not surprisingly in light of the teaching of proof, some students think the purpose of proof is to demonstrate that they can construct a sequence of statements that connects the truth of the premises to the truth of the target. Some of these students regard the truth of the target to be known beforehand. As the purpose of the proof is to exhibit their ability to produce an argument, it is not surprising that students say they never construct proofs unless they are assigned to do so. It is not surprising in this context, that students opt for a procedural approach, learning the parts, for example, of a proof by induction, learning to provide a proof for a base case, and learning to take a premise as a given and a conclusion of an implication to be proved. Some students do not understand why this procedure constructs a proof. Some express an unease -- they wish for the proof procedure to be convincing, to themselves. They are glad when they learn why the procedure does produce a convincing argument.

Some students recognize proof being used in class, for example in algorithms class and in introduction to the theory of computing.




Some students felt that proof was for finding out that a mathematical expression was true, or false. Some students knew that some statements could be proved undecidable.

\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/whyThemes}
\caption{Conceptualizations about why to study proofs}
\label{fig:whyThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline List of Known Facts & List of Warranted Facts & Warrants\\ 
\hline List of Warranted Facts & Means of Discovery & Tool rather than\\  & & demonstration to teachers\\ \hline
\end{tabular}
 \end{table}

\newpage
\section{What do students use proof for, when not assigned?}
Some students claim they never use proofs when not assigned.

It is not the case that any student, even when prompted, said they chose to carry out a proof without being directed to do so.
This could easily be due to a misunderstanding of the definition of proof.

\newpage
\section{Do students exhibit any consequence of inability in proof?}
Some students said that they knew how to craft recursive procedures, and enjoyed doing so when assigned problems designated as suitable for recursive implementations.
Some students said they did not employ recursive procedures in situations without a designation that recursive procedures were appropriate. They claimed not to be able to tell when recursive procedures were applicable.

\newpage
\section{What kind of structure do students notice in proofs?}
Some students think proofs are lists of statements without hierarchical structure.
Some students have asked what lemma means.
Some students knew that lemmas were built for use in larger proofs.
Some students were interested to hear about Dr. Lamport's structure in proofs.

\newpage
\section{What do students think it takes to make an argument valid?}
Some students, when prompted about rules of inference, felt that when all statement transformations were warranted, an argument was valid.
Some students stated that, when the target of the proof was true, the proof was valid, converse error.





We organize our overview of results beginning from an ideal Hilbert-axiomatic style of proof approach, and moving through approximations as they become greater departures from it.
\begin{figure}[h]
\centering
\includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/valid}
\caption{Conceptualizations about validity of proofs}
\label{fig:validityThemes}
\end{figure}
\begin{table}
 \caption{Critical factors for what a (valid) proof is}
\begin{tabular}{|c|c|c|}
\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline
\hline No Warrants & Some Warrants & Warrants\\ 
\hline Some Appropriate Warrants & Fully Warranted & thoroughness\\    \hline
\end{tabular}
 \end{table}

\paragraph{Definition based reasoning}
Some students, and some teaching assistants in their teaching, are not organizing the approach to proof around definitions.
Instead some students and some teaching assistants are focusing on an intuitive approach, involving examples.
Some students use examples to infer definitions.
Some students use single examples as proof.

Some students are not aware that proofs are illustrated with facts of, for the purposes of the class, less significance than the proof techniques. 
Some students are not aware of the relevance of proof to their intended career.
These students do not see any point to learning more than a procedural approach to the proof material, as they believe it to be of no lasting significance to them.

\paragraph{Generalization and transformation based reasoning}
Some students, and some instructors, do not emphasize that a single presentation can be seen as a representative of a group. For example, Mathematical  Association of America\cite{} publishes a proof of the Pythagorean Theorem that uses rectangles to illustrate that, when they are square, the Pythagorean Theorem is being shown to be true, though of course, the rectangles need not always be square. The proof, having been established, does not rely upon the rectangles remaining in a square condition.

 





\section{Combined Description}

There are a couple of ways students work with exercises in proof, that are incomplete.

Some students reason well with concrete entities, yet are confused with abstractions. These students are not appreciating the value of careful definitions, because they do not use them as tools, or the basis for reasoning. They are more comfortable with examples, because they are operating in a concrete world.

Some students do not connect the world of concrete objects with the abstract, symbolic representation, but are making use of symbols. Some operations transforming symbolic expressions are performed correctly, but not all. The lack of understanding of the symbols combined with a procedural approach to producing a proof artifact, leaves these students personally unconvinced, and unmotivated to make use of proof when it would be helpful to them.

Some students do understand application of facts, axioms and rules of inference, and are at home with careful definitions and symbolic concision. Some of these students also study math.


\newpage
\section{Diagram of Conceptualizations}

\begin{figure}[h]
\centering
\includegraphics[width=0.95\linewidth]{./themes}
\caption{Themes from interview data}
\label{fig:themes}
\end{figure}
\newpage
\section{Outcome Space}
The outcomes were not arranged in a single progression. Rather, there were several means, listed below, by which students were able to construct the proof artifact required by the class. The students did not always find the artifact convincing.

\begin{enumerate}
\item Concrete to Abstract -- generalize the argument, then the entities
\item Hilbert-style axiomatic/definitional proof
\item Abstract operations  -- symbols rather than entities, structure of argument
\end{enumerate}

The concrete to abstract path enabled students to reason with specific cases whose logic made sense to them, then make the step that the logical process itself was an entity that could be reused. The idea that other concrete entities could bear the same relationships, and be subject to the same reasoning constituted a step. The idea that analogies were being made, and that generalization was possible was another step.

The reasoning by axioms and rules of inference path was known to some students. These students mentioned their appreciation of math, and in some cases their discomfort with philosophy, in connection with symbolization and application of rules of inference.

One path was operation at the level of symbols, using procedures. This path is distinguished from that involving definitions, because some students, using definitions, were clear about appropriate operations to transform symbolic expressions, but students also sometimes were unsure about denotation and about appropriate operations.

\section{Critical Factors}
On the path from concrete to structured proofs, called herein "generalize the argument, then the entities", one critical factor is that an argument about one set of concrete entities can be used on another set, having analogous relationships.

Another critical factor is that, when an argument can be reused, sets of entities that stand in analogous relationships, the relationship can be generalized. When the relationship is generalized, the entities standing in that relationship can be given symbols.

On the path that starts with symbols, students have not generalized from concrete to abstract entities, rather, they have entered the fray at the level of abstraction of symbols. Thus, a critical factor is to understand the operations appropriate to the symbols, which imbues application of the rules of inference with significance. Another critical factor is that these symbols can represent entities of interest.





\section{Earlier Paper Material}

\subsection{Categories of Experience of Entering Students}
Undergraduate students beginning study of the computing disciplines present
various degrees of preparedness\cite{smith2013categorizing}. Some had no experience, some had had
informal experience, and some had had formal classes. The formal classes
extended from using applications to building applications. Informal experience
ranged from editing configuration files, such as background colors, to full time
jobs extended over multiple summers.
After publishing this paper, we encountered more related information. For
example, consistent with the work of Almstrum\cite{almstrum1996investigating}, we found that, about implications,
some students, who do know that any statement must and can, be
either true or false, thought implications must be true. Some interview participants
enjoyed a modified Moore method\cite{cohen1982modified} geometry class in middle school,
and relished opportunities to create proofs (not yet published). Other students
were not so well prepared.
\subsection{Representation/Symbolization in Pumping Lemmas}
We found that some students may lack facility in notation. For example, in the
application of the pumping lemma, students are expected to understand the
role of $i$, in the context that a string $s$, having component substrings $x$, $y$ and $z$,
can be used to generate other strings, of the form $xy^iz$, where $i$ gives the number
of copies of the substring $y$. Moreover, students are expected to understand that
the subdivision of a string of length $p$, $\sigma_1^a\sigma_2^{p-a}$, where $a \in \{0,1,\ldots,p\}$
uses $a$ as a parameter, a free variable, not one necessarily bound to a single instance
of a natural number, but a representation of a domain. An excerpt of the errors found on tests is shown in Table . Trigueros et
al. \cite[ p. 3]{jacobs2008developing} have observed that ``students are often unclear about the different
ways letters are used in mathematics''.  We have seen this
lack of understanding in a situation in which it was proposed as evidence that
a single example, namely $\sigma_1^a\sigma_2^{p-a}$, formed a proof for a universally quantified
statement.
Some of our results were consistent with the framework described by Harel
and Sowder in 1998\cite{harel1998students}. We found students holding conceptualizations that
Harel and Sowder's 1998 model calls symbolization. Harel and Sowder have
identified another category of conceptualization, that correctly applied transformation
and axiomatic arguments. Some students expressed enthusiasm for
the power that inheres to building arguments with carefully specified component
ideas, in particular how the absence of ambiguity permitted arguments to
extend to great length while remaining valid. Not all of the students had developed
axiomatic conceptualizations of proof. About definitions, we collected
preliminary data on students' conceptualizations of definitions used in proofs.
Some students thought definitions were boring. Some students thought that
they could infer definitions from a few examples. Concerning executive function,
we found that some students do not state the premises clearly, and some
students did not keep track of their goal. About rules of inference, we found
Figure 5.3.1: Some categories / conceptualizations found among students of
introduction to the theory of computing, and published at FIE.
that some students apply invalid approaches to inference.

\subsection{Abstract Model for Proof by Mathematical Induction and Recursion}
Far from finding agreement that (a) theorems are true as a consequence of
the definitions and the premise, and that (b) proofs serve to show how the
consequence is demonstrated from the premise, axioms and application of
rules of inference, instead we found a variety of notions about proof, including
the well-known procedural interpretation \cite{tall2008transition,weber2004traditional,tall2001symbols}, and the well-known empirical
misconception \cite{harel1998students}. The conceptualization that definitions are not necessarily
of interest compared with the procedures seemed different in kind from the
concept image / concept definition discoveries of R\"osken et al. \cite{rosken2007integrating}.
Interviews with students revealed that some students saw generation of a proof
by mathematic induction as a procedure to be followed, in which they should
produce a base case, and prove it, and should produce an induction step, and
prove that. This was consistent with Weber [?, p. 4-426] who has stated ``in
the studies that I conducted, it was more often the case that undergraduates
applied procedures that were not meaningful to them.'' He went on to give a
quotation from a participant [?, p. 4-426] ``And I prove something and I look at
it, and I thought, well, you know, it's been proved, but I still don't know that I
even agree with it [laughs]. I'm not convinced by my own proof!'' Some of the
students interviewed did not know why this procedure generated a convincing
argument. Polya[?] has written a problem involving all girls being blue-eyed;
a similar problem appears in Sipse \cite{sipser2012introduction} about all horses being the same color.
The purpose of this exercise is to make students aware that the truth of the
inductive step must apply when the base case appears as the premise. In some
cases, this point was not clear to the students.
Students' conceptualizations of proof by mathematical induction can support
their choice to apply recursive algorithms. One student reported success at
both mathematical induction and recursive algorithm application without ever
noticing any connection. This student opined that having learned recursion
with figures, and proof by mathematical induction without figures, that no
occasion for the information to spontaneously connect occurred. Students reporting
ability to implement assigned problems recursively, but not the ability
to understand proof by mathematical induction also reported that ability to
write recursive programs did not result in recognition of when recursive solutions
might be applicable in general. Students reporting ability to implement
assigned problems recursively, and also the ability to prove using mathematical
induction also reported preferring to implement recursive solutions in
problems as they arose.
Our work on students' choices of algorithmic approaches was consistent with
work by other researchers in computer science education\cite{booth1997phenomenography} on conceptualizations
of algorithms. Our work served to unify that of mathematician educators
with computer science educators, by providing a plausible explanation why
the conceptualizations of recursive algorithms that were found, might exist.
\subsection{Proofs by Induction}
Table 5.3.3: The Outcome Space for Proofs by Induction
Category Description
1Following procedure The method is learned, without understanding
2Understands base case The idea that a base case is proved by an existence
proof, often with a specific example
3Understands implication The idea that an implication is proved by
assuming the premise is not used
4Does not understand connection
Sees the implication and proves it well, but
does not anchor the succession to a base
case
5Does understand the argument
Understands the argument
6Knows why recursion
works
Can tailor the argument to explain recursive
algorithms
7Appreciates data structures
supporting recursion
Can see the benefit to algorithm from recursive
data structure
\subsection{Pumping Lemmas}
TABLE III. CATEGORIES\\
understand inequality\\
formulate correctly\\
distinguish between particular and generic particular\\
correctly apply universal quantifier\\
recognize string as member of language set

\section{Discussion}
 \subsection{Importance}
Importance goes here, rather than in analysis\\

Programmers/developers who produce and/or verify software that is used in safety critical applications, such as medical equipment, self-driving cars, and defense-related equipment should be able to know that their software functions correctly.

Programmers/developers who produce and/or verify software that is expected to perform work, such as search, efficiently, should be able to know that their algorithms are efficient.

Computer science is the expected background preparation for people working in these careers.
Proof is the method that is used to ascertain, and to convince, that these goals have been achieved.
\subsection{Interpretation of Results}
As in mathematics, some students learn as procedure that which we would prefer that they understand.
Some procedural learning is insufficiently accompanied by an understanding as to which contexts to which it applies, and has become in some cases what Whitehead calls "inert knowledge".

\subsection{More Discussion}
\paragraph{Helping Students Discern Abstraction}
Recall that variation theory holds that students cannot discern a thing unless
contrast is provided. Pang has pointed out that [], for persons aware of only
one language, ``speaking'' and ``speaking their language'' are conflated. Only
when the existence of a second language is known, does the idea of speaking
become separated from the idea of speaking a specific language.
(Here is a specialization (Hofstadter), formation of a new conjunct (Valiant), see Besold 2015)
Abstraction is important in computer science, and is worthy of investigation.
Inquiry into students' conceptualizations of formalization using symbols, symbolization,
has shown similar results among students of mathematics and of
computer science [?, ?]. Student populations contain the conceptualization that
proofs ought to be expressed using symbols, and some proof attempts show
that not all students are able to formalize meaningfully. Mathematics and computer
science pedagogies differ on the recommended style of variable names in
symbolization. In mathematics, there is a preference for single letter variable
names, and in computer science it is recognized that longer variable names assist
readers in understanding. In mathematics the use of single variable names
is preferred because it is thought to contribute to cultivating students' ability
to learn abstraction. If, in computer science education, we apply variation
Table 5.3.5: The Outcome Space for Proofs by Induction\\
Category Description\\
 1 Following procedure The method is learned, without understanding\\
  2 Understands base case The idea that a base case is proved by an existence
proof, often with a specific example\\
3 Understands implication The idea that an implication is proved by
assuming the premise is not used\\
4 Does not understand connection
Sees the implication and proves it well, but
does not anchor the succession to a base
case\\
5 Does understand the argument
Understands the argument\\
6 Knows why recursion
works\\
Can tailor the argument to explain recursive
algorithms\\
7 Appreciates data structures
supporting recursion\\
Can see the benefit to algorithm from recursive
data structure\\
theory, we gain confidence in the idea that students may discern the process
of abstraction as we vary the names of the variables. We could imagine deriving
code from a requirement about a specific class, and using corresponding
variable names, and we could show the process of promoting the code into a
more general class in the inheritance hierarchy, changing the variable names to
correspond to the more general domain of objects. Thus we can borrow from
the approach used by mathematics education, but make it more explicit, taking
advantage of computer science's explicit treatment of inheritance hierarchies in
object oriented code. Seeking evidence of students' conception of abstraction,
we could examine overridden methods to see whether variable names in more
and less general implementations bear that relation to one another.
\paragraph{Helping Students Discern Abstraction}
\paragraph {Algebra}
In middle or high school algebra students became familiar with the use of letters in equations,
and solving equations which resulted in individual values, or no value, being attached to the letters.

Ideally the ability to understand expressions, to formulate pre- and post conditions would be acquired.
As we have seen that this occurs sometimes, but does not always occur,
there may be benefit to some students to review this idea.
We might choose to emphasize abstraction in this process.

\paragraph{Geometry}
In high school geometry, formal proofs of geometric properties are covered.
Students are exposed to a form for argument, and are given examples of use of rules of inference to perform logical deduction.
We have seen that sometimes this process is appreciated in enough generality to be recognized
as an example of argumentation.
We have seen as well, that some students found this process entirely specific to geometry,
doubting that it had broader application.

\paragraph{Seeing a Broader Context}

It may be that some students do not see a separation between the activity of formalization on the one hand,
and the application area of finding solutions to equations on the other.
It may be that some students do not see a separation between the activity of deducing using logic on the one hand, and the application area of learning geometric facts on the other.
It seems consistent with neglecting opportunities for abstraction, that these separations are not always seen,
Speciation, an idea which uses abstraction, providing a hierarchy of properties animals and plants might have, was not recognized early or universally. So, it is
not surprising that abstraction, which involves choice about which details to defer, and which to regard as significant, is not always obvious.
In the machine learning perspective, features can be learned.
What do I want to say, it takes some effort to recognize features?
There might be a way to formulate choice of features such that some better efficiency is gained by thinking of the features in that order vs. another order.
(Such as, we never have to think about some features for some parts of the tree.)
If we think about knowledge being organized in neural networks, such that abstraction has a physical manifestation, we can see that ideas between which there is little distance (by some measure, neurons, glia?) in the tree will more frequently elicit one another by linkages at the metabolic level.
At the neural level, modifications for efficiency are constantly taking place (Do we have this from Kandel and Squire?).
We might wish to exploit this in the way we teach, to exhibit the abstraction deliberately, to minimize the amount of neural connection remodeling that would occur in the process of providing an efficient neural connection remodeling that would occur in the process of providing an efficient neural representation.
Being able to learn by analogy testifies to the utility of having a neural representation that corresponds to abstraction.
Students who are working without hierarchical organization of concepts are at a disadvantage.
\paragraph{Mathematics tests in high school that involve proving}
What can we learn from students of computer science who excelled in reasoning to this level?


\section{Previously Published Work}

Three papers in this area have been published to date:
\begin{itemize}

\item CCSCNE: Categorizing the School Experience of Entering Computing
Students
\item FIE: Mathematization in Teaching Pumping Lemmas
\item Koli Calling: Computer Science Students' Concepts of Proof by Induction
 
\end{itemize}
\section{ Categories of Experience of Entering Students}
Undergraduate students beginning study of the computing disciplines present
a various degrees of preparedness.\cite{reilly2014examination} Some interview participants enjoyed
a modified Moore method\cite{cohen1982modified} geometry class in middle school, and relished
opportunities to create proofs (not yet published). Other students are not so
well prepared.
After publishing this paper, more information relating to its topic has been
encountered. For example, consistent with the work of Almstrum \cite{almstrum1996investigating}, we have
found that, about implications, some students, who do know that any statement
must and can, be either true or false, think implications must be true.
\section{ Representation/Symbolization in Pumping Lemmas}
Some of our results to date are consistent with the framework described by
Harel and Sowder in 1998\cite{harel1998students}. We have found students holding conceptualizations
that Harel and Sowder's 1998 model\cite{harel1998students} calls symbolization: We have found
that some students may lack facility in notation. For example, in the application
of the pumping lemma, students are expected to understand the role of $i$,
in the context that a string $s$, having component substrings $x$, $y$ and $z$, can be
used to generate other strings, of the form $xy^iz$, where $i$ gives the number of
copies of the substring $y$. Moreover, students are expected to understand that
the subdivision of a string of length $p$, expressed as $\sigma_1^a\sigma_2^{p-a}$, where $a \in \{0,1,\ldots,p\}$
uses $a$ as a parameter, a free variable, not one necessarily bound to a single instance
of a natural number, but a representation of a domain. We have seen this
lack of understanding in a situation in which it was proposed as evidence that
a single example, namely $\sigma_1^a\sigma_2^{p-a}$, formed a proof for a universally quantified
statement. An excerpt of the errors found on tests is shown in Table .

Table : Some example errors\\
Let x be empty\\
$|xy| \leq p, so xy = 0^p$\\
$|xy| \leq p; let x = 0^{p+r}, y = 0^{p+r}, 0 < r < p$\\
Let's choose $|xy| = p$\\
$0^{p+1}0^b1^p \neq 0^{p+1}1^p \therefore xy^2z \not\in \mathcal{L}$\\
where $\mathcal{L} = \{0^i1^j, i \neq j\}$\\
we choose $s = 0^{p+1}1^p$ within $|xy|$\\
thus $\neq 0^p1^{p+1}$\\
Let $x = 0^a, y = 0^b1^a$\\
$x = 0^{p-h}, y = 0^h$\\
$x = 0^i, y = 0^i, z = 0^i1^j$

Figure: Some categories / conceptualizations found among students of
introduction to the theory of computing, and published at FIE.

Harel and Sowder identified a category of conceptualization that correctly
applied transformation and axiomatic arguments. Some students expressed
enthusiasm for the power that inheres to building arguments with carefully
specified component ideas, in particular how the absence of ambiguity permitted
arguments to extend to great length while remaining valid. Not all of the
students had developed axiomatic conceptualizations of proof. About definitions,
we have collected preliminary data on students' conceptualizations of
definitions used in proofs. Some students think definitions are boring. Some
students think that they can infer definitions from a few examples. Concerning
executive function, we have found that some students do not state the
premises clearly, and some students do not keep track of their goal. About
rules of inference, we have found that some students apply invalid approaches
to inference.
\section{ Abstract Model for Proof by Mathematical Induction and Recursion}
Interviews with students revealed that some students see generation of a proof
by mathematic induction as a procedure to be followed, in which they produce
a base case, and prove it, and produce an induction step, and prove that. Some
of the students interviewed did not know why this procedure generated a
convincing argument. Moore, as reported in Polya[] noted that some students
of mathematics formed the same conceptualization, that there is a procedure,
but it does not necessarily produce a convincing argument. Polya[] wrote
a problem involving all girls being blue-eyed; a similar problem appears in
Sipser\cite{sipser2012introduction} about all horses being the same color. The purpose of this exercise is
to make students aware that the truth of the inductive step must apply when
the base case appears as the premise. In some cases, this point was not clear to
the students.
Students' conceptualizations of proof by mathematical induction can support
their choosing to apply recursive algorithms. One student reported success at
both mathematical induction and recursive algorithm application without ever
noticing any connection. This student opined that having learned recursion
with figures, and proof by mathematical induction without figures, that no
occasion for the information to spontaneously connect occurred. Students reporting
ability to implement assigned problems recursively, but not the ability
to understand proof by mathematical induction also reported that ability to
write recursive programs did not result in recognition of when recursive solutions
might be applicable in general. Students reporting ability to implement
assigned problems recursively, and also the ability to prove using mathematical
induction also reported preferring to implement recursive solutions in
problems as they arose.
Our work on students' choices of algorithmic approaches is consistent with
work by other researchers in computer science education\cite{} on conceptualizations
of algorithms. Our work served to unify that of mathematician educators
with that of computer science educators, by providing a plausible explanation why
the conceptualizations of recursive algorithms that were found, might exist.

Figure 4.0.2: Conceptualizations of proof by induction and recursion, published
in Koli Calling

Index Element of Model
\begin{enumerate}

\item Some students begin learning proof by mathematical induction as if it were
a procedure.
\item Some students learn two parts of this proof technique without seeing any
connection between the two.
\item Some students do not find the procedure to be a convincing argument.
\item Some students would not employ proof by mathematical induction to explore
whether a recursive algorithm would apply to a given problem.
\item Some students understand both proof by mathematical induction and also
recursion and had never noticed any similarity.
 
\end{enumerate}
\section{Results of Combined Investigations}
There are some categories that are shared among the several contexts.
\section{Categories}
Categories found in one or more investigations

Categories\\
Definition of proof as convincing (to mathematicians) argument is not
always understood\\
Definitions in general are not always recognized as significant building
blocks in arguments\\
The idea of a false statement sometimes becomes troublesome when
negation is being learned.\\
In particular, accepting that an implication may be false, can be troublesome.
Notation is sometimes difficult.\\
Ideas presented relying on notation are not always connected with
ideas presented relying on figures.\\
Warrants are not always recognized.\\
Students do not always traverse levels of abstraction effectively.\\
The applicability of valid argument forms to contexts of interest is not
always appreciated.
\section{ Critical Factors}
To determine critical factors, we can convert negative categories into achievement
levels.
\begin{tabular}{p{3cm}p{3cm}}
Categories & Achievement Levels\\
The idea of a false statement
sometimes becomes troublesome
when negation is being
learned.&\\
&True and false make sense, and
we can make arguments using
them.\\
Definition of proof as convincing
argument is not always understood&\\
Warrants are not always recognized.&\\
&Proof can sometimes be obtained
through a series of warranted assertions.\\
Definitions in general are not always
recognized as significant
building blocks in arguments&\\
&Using agreed definitions and
valid rules of inference we can
sometimes explore the consequences
of definitions.\\
Notation is sometimes difficult.&\\
&Notation helps.\\
Ideas presented relying on notation
are not always connected
with ideas presented relying on
figures.&\\
&We might wish to help students
traverse multiple rendering of
ideas.\\
Students do not always traverse
levels of abstraction effectively.&\\
&We might wish to help students
traverse multiple levels of abstraction.\\
The applicability of valid argument
forms to contexts of interest
is not always appreciated.&\\
&We might wish to give exercise
with authentic (career related)
examples\\
 
\end{tabular}

Using the achievement levels we can infer critical factors.
\begin{tabular}{p{3cm}p{3cm}}
Achievement Levels& Critical Factors\\
True and false make sense, and
we can make arguments using
them.&\\
& True and false apply to assertions.\\
Proof can sometimes be obtained
through a series of warranted assertions.
& Proof is exploration and discovery.\\
Using agreed definitions and
valid rules of inference we can
sometimes explore the consequences
of definitions.&\\
& Efficiency but also abstraction
are aided by notation.\\
Notation helps.&\\
& Notation is one representation
and there are others. Ideas appear
in multiple guises.\\
We might wish to help students
traverse multiple rendering of
ideas.&\\
& When notation allows for multiple
interpretations, abstraction
above those multiple interpretations
has been achieved.\\
We might wish to help students
traverse multiple levels of abstraction.&\\
&Multiple levels of abstraction are
relevant at the same time.\\
We might wish to give exercise
with authentic (career related)
examples.&\\
& Authentic applications show the
use of this knowledge.\\
\end{tabular}


\subsection{Abstraction}
Literature reports \cite{} students of CS have trouble with abstraction.
Taking abstraction to be the ability to select some details to ignore,
and thereby find a simpler model of an entity, we can transform the ideal
knowledge transfer experience into on disabled by a lack of ability to see this dimension.
The multiple-inheritance hierarchy that could be used to organize
definitions and relationships of ideas is less able. More entities will be
grouped together than effective use of the multiple inheritance hierarchy
would consider equivalent.
Another useful concept that students have been seen to underappreciate is the significance of careful definitions.
Abstraction hierarchies allow for efficiency in definitions.
A new entity can be defined as a specialization of an existing entity, and its differences
make up the new definition material.
In the absence of this multiple inheritance hierarchy, every definition in its full length
is attached to its entity.
Tie in with Mazur. For students holding the same granularity of refinement of 
concepts, conversations would be easier, because there would be fewer disconnects as one participant expressed a thought on one degree of refinement far from that of another student.
If the ideas implying the refinement of the definition inheritance graph, being different from one
discussant to the next, are rare, and/or the meaning of the sentence does not depend upon it, these exchanges are not too disruptive or distressing.
On the other hand when two sets of refinement are very different,
and the meaning of the exchange depends upon the refinement in the speaker, that the  hearer does not have,
then some degree of failure of communication will ensue.
Absence of abstraction converts tree of topics into sequence of topics.
Tree of proof examples (say of application of proof technique) into sequence of examples.
Might detract from recognizing what is a related example.
Would detract from plausible inference technique of ``related problem seen before''.
We have a goal for student programming that they should strive for segments of programs
(e.g., method implementations) to be small. One way of accomplishing this is to use abstraction,
such as combining instructions into a method, and calling the method.
If students have difficulty with abstraction, they might
have difficulty with choosing groups of instructions to represent a method.
Correspondingly, if they practice grouping instruction into methods, and using those methods, they would 
be gaining practice relevant to using abstraction.
One way to cultivate abstraction is to pose a question of which one of several examples is different.
When several things are examples of one abstract idea and one is not, identifying the one that is different involves noticing the abstraction.
These questions could be instantiated using blocks of code.

\subsection{Definitions}
Without abstraction the burdensomeness of definition is increased. This could contribute to the reluctance of students to embrace definitions.
\subsection{Symbolization}
Use of symbols is a kind of abstraction.
Symbolization is the syntax for simple, clear definitions as Gries\cite[p.205]{gries2012science}(Science of Programming) recommends for construction of programs.
Students will be hindered at this program derivation/development style if symbolization is a not-yet-acquired skill.
Program development/derivation should begin with a formulation of the requirements.
Students may arrive with some programming experience that is of a more intuitive, less
mathematically disciplined sort.
We have to ask how we desire to cultivate the abilities of such students. 
Vygotsky discussed language acquisition by children, in which some children will have begun to invent
some terms for items in their environment, and will need to be guided to abandon neologisms for the naming generally agreed in their environment.
Kuhn discussed the reluctance of scientists who have been rewarded for operating in one
perspective on nature to adopt a different perspective.
Instructor may encounter a similar reluctance on the part of students
to adapt a scientifically/mathematically disciplined approach to programming,
especially if the students have experienced some success in their earlier work.
To win over such students,
demonstration of superior outcomes on problems, especially on problems that seem insoluble otherwise,
are more frequently convincing.
Happily, Professor Gries has provided such examples.
By showing superior relative efficacy of these approaches in an activity the students recognize as 
desirable, instructors could motivate the students to learn symbolization.
\subsection{Structure}
sequence vs. sequence that has come about from combining parts. Refer to Leslie Lamport's structure for proofs. Combine with Gries' proofs for deriving code. The purpose for getting through Goguen and Malcolm is that it applies to imperative programs.


\subsection{Assessing Validity}

The results have bridged papers in computer science education, by Professor Booth\cite{}, and mathematics education, by \cite{}.

The results have been consistent with results of others in mathematics education.