\chapter{Results} The results of a phenomenographic study comprise a set of categories of description of ways of experiencing (or capability for experiencing \cite[p. 126]{marton1997learning}) a phenomenon, and relations among those categories. Booth states\cite{booth2001learning} "The results are communicated as descriptions of the essential aspects of each category, illustrated by pertinent extracts from the data". In this section we give the results of the analysis: categories, a dimension of variation for the inclusion of component aspects of conceptualization, a dimension of variation in the nature of depth of understanding and critical aspects associated with the dimensions of variation. \section{Criteria for Results} Marton and Booth\cite[p. 125]{marton1997learning} give criteria for the quality of a set of descriptive categories. The collective experience, over all participants in the study should be included. The individual categories should each stand in clear relation to the phenomenon of the investigation so that each category tells us something distinct about a particular way of experiencing the phenomenon. The categories have to stand in a logical relationship with one another, a relationship that is frequently hierarchical. Finally the systems should be parsimonious, which is to say that as few categories should be explicated as is feasible and reasonable, for capturing the critical variation in the data. late stages of analysis able to see aspects/facets of research object, see how they fit together like jigsaw pieces, see it against background, and communicate it to others. There are results for each of the research questions, and some combined results. While mainly we are discussing proof in general, it can help to think about one proof at a time. Applying the analytical framework of Marton and Booth\cite[p.43]{marton1997learning} \begin{table}[placement] \caption{Outcome space for what is proof, with temporal facet} \begin{tabular}{|p{4cm}|p{4cm}|p{4cm}|}\hline Acquiring & Knowing & Making Use of\\\hline\hline see the steps & remembering the steps & write it out\\\hline understand the steps & remembering the meaning & produce the meaning\\\hline understand steps and warrants & understanding the meaning & be able to apply the proof to simple examples\\\hline analyzed the structure, determine the warrants & understand the relevance & be able to apply the proof in general, know its context of applicability\\\hline \end{tabular} \end{table} Our goal might be in the lower right, and for some students who do not arrive that far, they might arrive at any of its three neighbors in the chart. We are looking for ways of experiencing, for example, one way is, proofs only apply to number facts, vs. proof techniques are separable from number facts and can be used on other domains. \section{What do students think a proof is?} The categories of description for what students think a proof is are listed in Table \ref{table:catsQ1}. \begin{table} \label{table:catsQ1} \caption{Categories of Description for: what a proof is} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline List of Known Facts & List of Warranted Facts & Warrants\\ \hline List of Warranted Facts & Means of Discovery & Tool rather than\\ & & demonstration to teachers\\ \hline \end{tabular} \end{table} The dimension of variation for aspect inclusion is illustrated in Table \ref{dim1Q1}. \begin{table} \label{dim1Q1} \caption{Dimension of Variation for Component Inclusion for: what a proof is} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline Attempts at statements & Statements (well-formulated) & mathematical formulation\\ \hline Statements (well-formulated) & List of Warranted Statements & Warrants\\ \hline List of Warranted Statements & Process of Transformation & purposive change\\ \hline Process of Transformation & Arrival at Goal & problem-solving\\\hline \end{tabular} \end{table} The dimension of variation for depth of understanding is illustrated in Table \ref{dim2Q1}. \begin{table} \label{dim2Q1} \caption{Dimension of Variation for Depth of Understanding for: what a proof is} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline procedure Including steps & class of allowed steps where warrants control membership and definitions control availability of warrants & Definitions and Warrants\\ \hline class of allowed steps where warrants control membership and definitions control availability of warrants & structure and subgoals & divide and conquer\\ & & demonstration to teachers\\ \hline structure and subgoals & abstraction & analogy making, generalization \\\hline \end{tabular} \end{table} The critical factors for components of what students think a proof is are listed in Table \ref{cf1Q1} \begin{table} \label{cf1Q1} \caption{Critical Factors for Component Inclusion for: what a proof is} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ & & \\ \hline \end{tabular} \end{table} The critical factors for depth of understanding of what students think a proof is are listed in Table \ref{cf2Q1}. \begin{table} \label{cf2Q1} \caption{Critical Factors for Depth of Understanding for: what a proof is} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ & & \\ \hline \end{tabular} \end{table} Some students, when asked what they think a proof is, will report that they think it is a list of true mathematically formulated statements, demonstrating the truth of a mathematically formulated statement. Some students report that a proof has a goal, a statement to be proved true. Some students know that the identified goal does not have to be known to be true in advance of the first proof. Sometimes, though, students have the idea that the proof is an exhibit of their ability to connect known facts, including the goal as a known fact. Some students, for example some taking philosophy, understand a proof more generally as not having a requirement for a mathematical formulation. Some of these have expressed dislike of such less precisely articulated statements. Some students, when prompted, will acknowledge that warrants for these statements are required. Axioms and agreed facts do not require warrants. Some students, but not all, recognize that premises do not require warrants. Some students, but not all, recognize that suppositions, as premises, do not require warrants. Some students, but not all, recognize that cases, as suppositions, do not require warrants. Some students, but not all, know that progress from one statement to the next, a transformation of a statement, requires a warrant. Some of the optional syntactic ornamentation of a proof, such as literal text "Proof:", and "QED" or $\qed$, are used by some students as proxies for the proof. As in the research by Harel and Sowder\cite{harel1998students}, which they describe as "ritual proof", we find in our research that some students claim to recognize a proof when they see these artifacts, and claim they have not seen a proof when they do not see these artifacts. Some students are aware that proof, as encountered in class, ought to be a convincing argument. These students feel that something is wrong when they are not convinced by the proof technique they have learned to execute in a procedural fashion. Some students know that proof is convincing others, and also ascertaining for oneself. Of these, some find that proof is convincing for some facts they regard as mathematical, yet do not think proof is applicable to programs as large as those with which they plan to be involved. Some of these students have not yet acquired the perspective that proving theorems about the number of instruction executions, and/or memory locations needed are both numerical and also applicable to and relevant for software development. \begin{figure}[h] \centering \includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/whatThemes} \caption{Conceptualizations found for what a proof is} \label{fig:whatThemes1} \end{figure} %\begin{table} % \caption{Critical factors for what a proof is} %\begin{tabular}{|c|c|c|} %\hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline %\hline List of Known Facts & List of Warranted Facts & Warrants\\ %\hline List of Warranted Facts & Means of Discovery & Tool rather than\\ & & demonstration to teachers\\ \hline %\end{tabular} % \end{table} \newpage \section{How do students approach understanding a proof?} The categories of description for How do students approach understanding are listed in Table \ref{catsQ2}. \begin{table} \label{catsQ2} \caption{Categories of Description for: How do students approach understanding} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for aspect inclusion is illustrated in Table \ref{dim1Q2}. \begin{table} \label{dim1Q2} \caption{Dimension of Variation for Component Inclusion for: How do students approach understanding} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline List of Known Facts & List of Warranted Facts & Warrants\\ \hline List of Warranted Facts & Means of Discovery & Tool rather than\\ & & demonstration to teachers\\ \hline \end{tabular} \end{table} The dimension of variation for depth of understanding is illustrated in Table \ref{dim2Q2}. \begin{table} \label{dim2Q2} \caption{Dimension of Variation for Depth of Understanding for: How do students approach understanding} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline List of Known Facts & List of Warranted Facts & Warrants\\ \hline List of Warranted Facts & Means of Discovery & Tool rather than\\ & & demonstration to teachers\\ \hline \end{tabular} \end{table} The critical factors for components of How do students approach understanding are listed in Table \ref{cf1Q2} \begin{table} \label{cf1Q2} \caption{Critical Factors for Component Inclusion for: How do students approach understanding} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline List of Known Facts & List of Warranted Facts & Warrants\\ \hline List of Warranted Facts & Means of Discovery & Tool rather than\\ & & demonstration to teachers\\ \hline \end{tabular} \end{table} The critical factors for depth of understanding of How do students approach understanding are listed in Table \ref{cf2Q2}. \begin{table} \label{cf2Q2} \caption{Critical Factors for Depth of Understanding for: How do students approach understanding} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline List of Known Facts & List of Warranted Facts & Warrants\\ \hline List of Warranted Facts & Means of Discovery & Tool rather than\\ & & demonstration to teachers\\ \hline \end{tabular} \end{table} Those students who felt they understood some proofs approached them by checking whether they felt each line of a proof was true. Some of these mentioned that a statement should be warranted by previous statements. \begin{figure}[h] \centering \includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/howThemes} \caption{Student Conceptualizations of How to Comprehend a Proof} \label{fig:howThemes2} \end{figure} \newpage \section{What do students think a proof is for?} The categories of description for what students think a proof is for are listed in Table \ref{catsQ3}. \begin{table} \label{catsQ3} \caption{Categories of Description for: What a proof is for} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline Pointless & Understand proofs about resource utilization & \\ \hline Understand proofs about resource utilization & Demonstrate correctness of previously designed algorithms & \\ \hline Demonstrate correctness of previously designed algorithms & Show where modification to algorithm might occur to be proved & \\ \hline Show where modification to algorithm might occur to be proved & Explore possibilities for algorithms & \\ \hline \end{tabular} \end{table} The dimension of variation for component inclusion is illustrated in Table \ref{dim1Q3}. \begin{table} \label{dim1Q3} \caption{Dimension of Variation for Component Inclusion for: what a proof is for} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline checking algorithms (ascertaining) & explaining algorithms (persuading) & \\ \hline explaining algorithms (persuading) & designing algorithms & \\ \hline \end{tabular} \end{table} The dimension of variation for depth of understanding is illustrated in Table \ref{dim2Q3}. \begin{table} \label{dim2Q3} \caption{Dimension of Variation for Depth of Understanding for: what a proof is for } \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline reiterate what is known & clarifying what was not obvious & \\ \hline clarifying what was not obvious & exploration of possibilities of improvement (bounds) & \\ \hline exploration of possibilities of improvement (bounds) & guiding algorithm development &\\ \hline \end{tabular} \end{table} The critical factors for components of what students think a proof is for are listed in Table \ref{cf1Q3} \begin{table} \label{cf1Q3} \caption{Critical Factors for Component Inclusion for: what a proof is for } \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & Warrants\\ \hline & & \\ \hline \end{tabular} \end{table} special purpose tool\\ general purpose transformation of representation, to elucidate connections between ideas\\ means of translation to most helpful representation The critical factors for depth of understanding of what students think a proof is are listed in Table \ref{cf2Q3}. \begin{table} \label{cf2Q3} \caption{Critical Factors for Depth of Understanding for: what a proof is for} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} Some students think that proofs are not applicable to what they do. They think they do not need to know it. Because they do not need to know it, they logically conclude that learning to produce a "proof" procedurally is enough, because, it earns full credit. Some students combine the learning about proof with the subject matter that is used to exercise proof techniques; they think proof is for demonstrating facts about numbers. Some students claim that they never produce proofs unless assigned to do so in class. Let us call the statement to be proved, the target, so as to more clearly articulate the variety of student thinking, by escaping the connotations of "statement to be proved". Perhaps not surprisingly in light of the teaching of proof, some students think the purpose of proof is to demonstrate that they can construct a sequence of statements that connects the truth of the premises to the truth of the target. Some of these students regard the truth of the target to be known beforehand. As the purpose of the proof is to exhibit their ability to produce an argument, it is not surprising that students say they never construct proofs unless they are assigned to do so. It is not surprising in this context, that students opt for a procedural approach, learning the parts, for example, of a proof by induction, learning to provide a proof for a base case, and learning to take a premise as a given and a conclusion of an implication to be proved. Some students do not understand why this procedure constructs a proof. Some express an unease -- they wish for the proof procedure to be convincing, to themselves. They are glad when they learn why the procedure does produce a convincing argument. Some students recognize proof being used in class, for example in algorithms class and in introduction to the theory of computing. Some students felt that proof was for finding out that a mathematical expression was true, or false. Some students knew that some statements could be proved undecidable. \begin{figure}[h] \centering \includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/whyThemes} \caption{Conceptualizations about why to study proofs} \label{fig:whyThemes3} \end{figure} \newpage \section{What do students use proof for, when not assigned?} The categories of description for what students do students use proof for, when not assigned? are listed in Table \ref{catsQ4}. \begin{table} \label{catsQ4} \caption{Categories of Description for: do students use proof for, when not assigned?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for component inclusion is illustrated in Table \ref{dim1Q4}. \begin{table} \label{dim1Q4} \caption{Dimension of Variation for Component Inclusion for: do students use proof for, when not assigned?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for depth of understanding is illustrated in Table \ref{dim2Q4}. \begin{table} \label{dim2Q4} \caption{Dimension of Variation for Depth of Understanding for: do students use proof for, when not assigned?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for components of do students use proof for, when not assigned? are listed in Table \ref{cf1Q4} \begin{table} \label{cf1Q4} \caption{Critical Factors for Component Inclusion for: do students use proof for, when not assigned?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for depth of understanding of do students use proof for, when not assigned? are listed in Table \ref{cf2Q4}. \begin{table} \label{cf2Q4} \caption{Critical Factors for Depth of Understanding for: do students use proof for, when not assigned?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} Some students claim they never use proofs when not assigned. It is not the case that any student, even when prompted, said they chose to carry out a proof without being directed to do so. This could easily be due to a misunderstanding of the definition of proof. \newpage \section{Do students exhibit any consequence of inability in proof?} The categories of description for Do students exhibit any consequence of inability in proof? are listed in Table \ref{catsQ5}. \begin{table} \label{catsQ5} \caption{Categories of Description for: Do students exhibit any consequence of inability in proof?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for component inclusion is illustrated in Table \ref{dim1Q5}. \begin{table} \label{dim1Q5} \caption{Dimension of Variation for Component Inclusion for: Do students exhibit any consequence of inability in proof?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for depth of understanding is illustrated in Table \ref{dim2Q5}. \begin{table} \label{dim2Q5} \caption{Dimension of Variation for Depth of Understanding for: Do students exhibit any consequence of inability in proof?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for components of Do students exhibit any consequence of inability in proof? are listed in Table \ref{cf1Q5} \begin{table} \label{cf1Q5} \caption{Critical Factors for Component Inclusion for: Do students exhibit any consequence of inability in proof?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for depth of understanding of what students think a proof is are listed in Table \ref{cf2Q5}. \begin{table} \label{cf2Q5} \caption{Critical Factors for Depth of Understanding for: Do students exhibit any consequence of inability in proof?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} Some students said that they knew how to craft recursive procedures, and enjoyed doing so when assigned problems designated as suitable for recursive implementations. Some students said they did not employ recursive procedures in situations without a designation that recursive procedures were appropriate. They claimed not to be able to tell when recursive procedures were applicable. \newpage \section{What kind of structure do students notice in proofs?} The categories of description for What kind of structure do students notice in proofs? are listed in Table \ref{catsQ6}. \begin{table} \label{catsQ6} \caption{Categories of Description for: What kind of structure do students notice in proofs?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for component inclusion is illustrated in Table \ref{dim1Q6}. \begin{table} \label{dim1Q6} \caption{Dimension of Variation for Component Inclusion for: What kind of structure do students notice in proofs?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline List of Known Facts & List of Warranted Facts & Warrants\\ \hline List of Warranted Facts & Means of Discovery & Tool rather than\\ & & demonstration to teachers\\ \hline \end{tabular} \end{table} The dimension of variation for depth of understanding is illustrated in Table \ref{dim2Q6}. \begin{table} \label{dim2Q6} \caption{Dimension of Variation for Depth of Understanding for: What kind of structure do students notice in proofs?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & Warrants\\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for components of What kind of structure do students notice in proofs? are listed in Table \ref{cf1Q6} \begin{table} \label{cf1Q6} \caption{Critical Factors for Component Inclusion for: What kind of structure do students notice in proofs?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for depth of understanding of What kind of structure do students notice in proofs? are listed in Table \ref{cf2Q6}. \begin{table} \label{cf2Q6} \caption{Critical Factors for Depth of Understanding for: What kind of structure do students notice in proofs?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} Some students think proofs are lists of statements without hierarchical structure. Some students have asked what lemma means. Some students knew that lemmas were built for use in larger proofs. Some students were interested to hear about Dr. Lamport's structure in proofs. \newpage \section{What do students think it takes to make an argument valid?} The categories of description for What do students think it takes to make an argument valid? are listed in Table \ref{catsQ7}. \begin{table} \label{catsQ7} \caption{Categories of Description for: What do students think it takes to make an argument valid?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for component inclusion is illustrated in Table \ref{dim1Q7}. \begin{table} \label{dim1Q7} \caption{Dimension of Variation for Component Inclusion for: What do students think it takes to make an argument valid?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The dimension of variation for depth of understanding is illustrated in Table \ref{dim2Q7}. \begin{table} \label{dim2Q7} \caption{Dimension of Variation for Depth of Understanding for: What do students think it takes to make an argument valid?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for components of What do students think it takes to make an argument valid? are listed in Table \ref{cf1Q7} \begin{table} \label{cf1Q7} \caption{Critical Factors for Component Inclusion for: What do students think it takes to make an argument valid?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} The critical factors for depth of understanding of What do students think it takes to make an argument valid? are listed in Table \ref{cf2Q7}. \begin{table} \label{cf2Q7} \caption{Critical Factors for Depth of Understanding for: What do students think it takes to make an argument valid?} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline & & \\ \hline & & \\ \hline \end{tabular} \end{table} Some students, when prompted about rules of inference, felt that when all statement transformations were warranted, an argument was valid. Some students stated that, when the target of the proof was true, the proof was valid, converse error. We organize our overview of results beginning from an ideal Hilbert-axiomatic style of proof approach, and moving through approximations as they become greater departures from it. \begin{figure}[h] \centering \includegraphics[width=0.7\linewidth]{/home/theresesmith/Documents/2015Fall/Research/Thesis/valid} \caption{Conceptualizations about validity of proofs} \label{fig:validityThemes1} \end{figure} \begin{table} \caption{Critical factors for what a (valid) proof is} \begin{tabular}{|c|c|c|} \hline Lesser Conceptualization & Better Conceptualization & Critical Factor\\ \hline \hline No Warrants & Some Warrants & Warrants\\ \hline Some Appropriate Warrants & Fully Warranted & thoroughness\\ \hline \end{tabular} \end{table} \paragraph{Definition based reasoning} Some students, and some teaching assistants in their teaching, are not organizing the approach to proof around definitions. Instead some students and some teaching assistants are focusing on an intuitive approach, involving examples. Some students use examples to infer definitions. Some students use single examples as proof. Some students are not aware that proofs are illustrated with facts of, for the purposes of the class, less significance than the proof techniques. Some students are not aware of the relevance of proof to their intended career. These students do not see any point to learning more than a procedural approach to the proof material, as they believe it to be of no lasting significance to them. \paragraph{Generalization and transformation based reasoning} Some students, and some instructors, do not emphasize that a single presentation can be seen as a representative of a group. For example, Mathematical Association of America\cite{} publishes a proof of the Pythagorean Theorem that uses rectangles to illustrate that, when they are square, the Pythagorean Theorem is being shown to be true, though of course, the rectangles need not always be square. The proof, having been established, does not rely upon the rectangles remaining in a square condition. \section{Combined Description} There are a couple of ways students work with exercises in proof, that are incomplete. One of the groups of conceptualizations, which is consistent with that observed by Harel and Sowder\cite{harel1998students} and others (OK, get some more citations), it that a proof-attempt can be the result of filling in a template, and it can earn full points, without providing assurance to the person who enacted the procedure and filled in the template. Some students receive full credit, and some receive partial credit without explanation. Some students do not further pursue either situation (full credit without internal conviction, or partial credit) because they say they do not find the time; those who do not know why proof is taught in the curriculum might assign a low priority to following up these situations. Some students reason well with concrete entities, yet are confused with abstractions. These students are not appreciating the value of careful definitions, because they do not use them as tools, or the basis for reasoning. They are more comfortable with examples, because they are operating in a concrete world. Some students do not connect the world of concrete objects with the abstract, symbolic representation, but are making use of symbols. Some operations transforming symbolic expressions are performed correctly, but not all. The lack of understanding of the symbols combined with a procedural approach to producing a proof artifact, leaves these students personally unconvinced, and unmotivated to make use of proof when it would be helpful to them. Some students do understand application of facts, axioms and rules of inference, and are at home with careful definitions and symbolic concision. Some of these students also study math. \newpage \section{Diagram of Conceptualizations} \begin{figure}[h] \centering \includegraphics[width=0.95\linewidth]{./themes} \caption{Themes from interview data} \label{fig:themes} \end{figure} \newpage \section{Outcome Space} The outcomes were not arranged in a single progression. Rather, there were several means, listed below, by which students were able to construct the proof artifact required by the class. The students did not always find the artifact convincing. \begin{enumerate} \item Concrete to Abstract -- generalize the argument, then the entities \item Hilbert-style axiomatic/definitional proof \item Abstract operations -- symbols rather than entities, structure of argument \end{enumerate} The concrete to abstract path enabled students to reason with specific cases whose logic made sense to them, then make the step that the logical process itself was an entity that could be reused. The idea that other concrete entities could bear the same relationships, and be subject to the same reasoning constituted a step. The idea that analogies were being made, and that generalization was possible was another step. The reasoning by axioms and rules of inference path was known to some students. These students mentioned their appreciation of math, and in some cases their discomfort with philosophy, in connection with symbolization and application of rules of inference. One path was operation at the level of symbols, using procedures. This path is distinguished from that involving definitions, because some students, using definitions, were clear about appropriate operations to transform symbolic expressions, but students also sometimes were unsure about denotation and about appropriate operations. \section{Critical Factors} On the path from concrete to structured proofs, called herein "generalize the argument, then the entities", one critical factor is that an argument about one set of concrete entities can be used on another set, having analogous relationships. Another critical factor is that, when an argument can be reused, sets of entities that stand in analogous relationships, the relationship can be generalized. When the relationship is generalized, the entities standing in that relationship can be given symbols. On the path that starts with symbols, students have not generalized from concrete to abstract entities, rather, they have entered the fray at the level of abstraction of symbols. Thus, a critical factor is to understand the operations appropriate to the symbols, which imbues application of the rules of inference with significance. Another critical factor is that these symbols can represent entities of interest. \section{Earlier Paper Material} \subsection{Categories of Experience of Entering Students} Undergraduate students beginning study of the computing disciplines present various degrees of preparedness\cite{smith2013categorizing}. Some had no experience, some had had informal experience, and some had had formal classes. The formal classes extended from using applications to building applications. Informal experience ranged from editing configuration files, such as background colors, to full time jobs extended over multiple summers. After publishing this paper, we encountered more related information. For example, consistent with the work of Almstrum\cite{almstrum1996investigating}, we found that, about implications, some students, who do know that any statement must and can, be either true or false, thought implications must be true. Some interview participants enjoyed a modified Moore method\cite{cohen1982modified} geometry class in middle school, and relished opportunities to create proofs (not yet published). Other students were not so well prepared. \subsection{Representation/Symbolization in Pumping Lemmas} We found that some students may lack facility in notation. For example, in the application of the pumping lemma, students are expected to understand the role of $i$, in the context that a string $s$, having component substrings $x$, $y$ and $z$, can be used to generate other strings, of the form $xy^iz$, where $i$ gives the number of copies of the substring $y$. Moreover, students are expected to understand that the subdivision of a string of length $p$, $\sigma_1^a\sigma_2^{p-a}$, where $a \in \{0,1,\ldots,p\}$ uses $a$ as a parameter, a free variable, not one necessarily bound to a single instance of a natural number, but a representation of a domain. An excerpt of the errors found on tests is shown in Table . Trigueros et al. \cite[ p. 3]{jacobs2008developing} have observed that students are often unclear about the different ways letters are used in mathematics''. We have seen this lack of understanding in a situation in which it was proposed as evidence that a single example, namely $\sigma_1^a\sigma_2^{p-a}$, formed a proof for a universally quantified statement. Some of our results were consistent with the framework described by Harel and Sowder in 1998\cite{harel1998students}. We found students holding conceptualizations that Harel and Sowder's 1998 model calls symbolization. Harel and Sowder have identified another category of conceptualization, that correctly applied transformation and axiomatic arguments. Some students expressed enthusiasm for the power that inheres to building arguments with carefully specified component ideas, in particular how the absence of ambiguity permitted arguments to extend to great length while remaining valid. Not all of the students had developed axiomatic conceptualizations of proof. About definitions, we collected preliminary data on students' conceptualizations of definitions used in proofs. Some students thought definitions were boring. Some students thought that they could infer definitions from a few examples. Concerning executive function, we found that some students do not state the premises clearly, and some students did not keep track of their goal. About rules of inference, we found Figure 5.3.1: Some categories / conceptualizations found among students of introduction to the theory of computing, and published at FIE. that some students apply invalid approaches to inference. \subsection{Abstract Model for Proof by Mathematical Induction and Recursion} Far from finding agreement that (a) theorems are true as a consequence of the definitions and the premise, and that (b) proofs serve to show how the consequence is demonstrated from the premise, axioms and application of rules of inference, instead we found a variety of notions about proof, including the well-known procedural interpretation \cite{tall2008transition,weber2004traditional,tall2001symbols}, and the well-known empirical misconception \cite{harel1998students}. The conceptualization that definitions are not necessarily of interest compared with the procedures seemed different in kind from the concept image / concept definition discoveries of R\"osken et al. \cite{rosken2007integrating}. Interviews with students revealed that some students saw generation of a proof by mathematic induction as a procedure to be followed, in which they should produce a base case, and prove it, and should produce an induction step, and prove that. This was consistent with Weber [?, p. 4-426] who has stated in the studies that I conducted, it was more often the case that undergraduates applied procedures that were not meaningful to them.'' He went on to give a quotation from a participant [?, p. 4-426] And I prove something and I look at it, and I thought, well, you know, it's been proved, but I still don't know that I even agree with it [laughs]. I'm not convinced by my own proof!'' Some of the students interviewed did not know why this procedure generated a convincing argument. Polya[?] has written a problem involving all girls being blue-eyed; a similar problem appears in Sipse \cite{sipser2012introduction} about all horses being the same color. The purpose of this exercise is to make students aware that the truth of the inductive step must apply when the base case appears as the premise. In some cases, this point was not clear to the students. Students' conceptualizations of proof by mathematical induction can support their choice to apply recursive algorithms. One student reported success at both mathematical induction and recursive algorithm application without ever noticing any connection. This student opined that having learned recursion with figures, and proof by mathematical induction without figures, that no occasion for the information to spontaneously connect occurred. Students reporting ability to implement assigned problems recursively, but not the ability to understand proof by mathematical induction also reported that ability to write recursive programs did not result in recognition of when recursive solutions might be applicable in general. Students reporting ability to implement assigned problems recursively, and also the ability to prove using mathematical induction also reported preferring to implement recursive solutions in problems as they arose. Our work on students' choices of algorithmic approaches was consistent with work by other researchers in computer science education\cite{booth1997phenomenography} on conceptualizations of algorithms. Our work served to unify that of mathematician educators with computer science educators, by providing a plausible explanation why the conceptualizations of recursive algorithms that were found, might exist. \subsection{Proofs by Induction} Table 5.3.3: The Outcome Space for Proofs by Induction Category Description 1Following procedure The method is learned, without understanding 2Understands base case The idea that a base case is proved by an existence proof, often with a specific example 3Understands implication The idea that an implication is proved by assuming the premise is not used 4Does not understand connection Sees the implication and proves it well, but does not anchor the succession to a base case 5Does understand the argument Understands the argument 6Knows why recursion works Can tailor the argument to explain recursive algorithms 7Appreciates data structures supporting recursion Can see the benefit to algorithm from recursive data structure \subsection{Pumping Lemmas} TABLE III. CATEGORIES\\ understand inequality\\ formulate correctly\\ distinguish between particular and generic particular\\ correctly apply universal quantifier\\ recognize string as member of language set