ch4.tex

\chapter{Phenomenographic Analysis}
 
%\section{Application of Phenomenographic Analysis in this Study}
 
 We applied phenomenographic analysis to transcripts, field notes and documents. We addressed several research questions. The analyses are organized herein by the question addressed.
 
 The analysis for the research question "What do students think proof is for?", which was approached as "Why do you think we teach proof?" exemplifies a phenomenographic approach. One aspect of the phenomenon of proof is its utility. We selected student verbal productions related to the use of proof. We considered them in the context of their own interview, and we compared them to data from other interviews on the same theme.
 
 The questions are ordered guided by the 1956 version of Bloom's Taxonomy, namely, recognition, comprehension, application, analysis, synthesis, evaluation.\cite{bloom1956taxonomy}
 
 % % %recognition
 
\section{Phenomenographic Analysis of What Students Think Proofs Are}
  
  The categories developed in the traditional phenomenographic analysis are:
  
  \begin{table}
  \caption{Categories for Student Conceptualizations of What Proofs Are}
  \begin{tabular}{|p{6cm}|p{6cm}|}\hline
  Category & Description\\\hline\hline
   Make claims obviously correct & \\\hline
    Arguments in support of an idea or claim & \\\hline
     Combinations of Standard Argument Forms & \\\hline
      Composed of Mathematical Statements & \\\hline
       Contain Certain Syntactic Elements & \\\hline
  Element of Domain of Mental Constructs & \\\hline
  \end{tabular}
  \end{table}
  
  Ideas that would have been welcome but did not appear: 
  
   \begin{table}
   \caption{Relationships for  Student Conceptualizations of What Proofs Are}
    \begin{tabular}{|p{6cm}|p{6cm}|}\hline
    Idea & Description\\\hline\hline
    Consequence of Definitions & \\\hline
    Relationship to Examples & \\\hline
    Relative Value vs. Experiment & we do not expect students to say that proofs wouldn't entirely replace experimentation, but would back up experiment. We prefer students would know that proof is sufficient on its own.\\\hline
    \end{tabular}
    \end{table}
    
The arrangement of the categories follows that of the table, and is shown in Figure \ref{fig:WhatProof}.
    
\begin{figure}
\centering
\includegraphics[width=0.7\linewidth]{./WhatProof}
\caption{Categories from what is proof}
\label{fig:WhatProof}
\end{figure}


Following the traditional phenomenographic method, we exam pairs of categories. We choose categories that appear to be adjacent in the space of features with which categories are distinguished from one another. This calls attention to features whose values differ. These differing values are candidates for critical aspects. We can consider whether a particular difference in feature value is important in distinguishing one category of conceptualization from another. The confidence with which we hold that difference in feature value to be important is the confidence with which we feel that difference is a critical aspect.

For example, there is a conceptualization found in the cohort of students that in a spoken proof attempt, will produce the phrase "You know what I mean." The aspect of argumentation that certain forms, such as mathematical formulation, can be suitable for proof, and other forms, such as "You know what I mean." are not, in general, appropriate seems very important. We propose that "express ideas with logical statements, including mathematical formulation" is a critical aspect differentiating the category "Domain of Mental Constructs" from "Composed of Statements".
Critical aspects are listed in Table \ref{tab:critWhatIs}.

\begin{table}[ht]
\caption{Critical factors for What is proof?}
\label{tab:critWhatIs}
\begin{tabular}{|p{8cm}|}\hline
Critical Aspect\\ \hline\hline
Express ideas with logical statements, including mathematical formulation.\\ \hline
Later statements must be justified by what has gone before.\\ \hline
A guiding principle in the development of the argument is to reach the state in which the desired claim has been made obviously true, or false.\\ \hline
One criterion for evaluating an argument is that it renders the claim obviously true or false.\\ \hline
\end{tabular}
\end{table}

Considering the categories "Composed of Statements" and "Combination of Standard Argument Forms", the aspect of relationships between statements, specifically that later statements must be justified by what has gone before, seems critical.

Considering the categories "Combination of Standard Argument Forms" and "Argument in Support of an Idea or Claim", the proposed critical aspect is that guidance about the development of the argument comes from the goal claim. There is evidence of students setting out to create an argument, but getting lost, such as forgetting that a scope has been set in which a statement is temporarily held to be false, and forgetting to exit that scope.

Considering the categories "Argument in Support of Idea or Claim" and "Makes a Claim Obviously Correct", we propose that the rendering obvious, of the claim, by the argument, is the most important distinguishing feature.
  
  Carnap writes eloquently on proof, a subset of logical deduction:
  \begin{quote}
  The essential character of logical deduction, i.e. concluding from a sentence $\mathfrak{S}_i$ a sentence $\mathfrak{S}_j$ that is L-implied by it, consists in the fact that the content of $\mathfrak{S}_j$  is contained in the content of $\mathfrak{S}_i$ (because the range of $\mathfrak{S}_i$ is contained in that of $\mathfrak{S}_j$). We see thereby that logical deduction can never provide us with new knowledge about the world. In every deduction the range either enlarges or remains the same, which is to say the content either diminishes or remains the case \textit{Content can never be increased by a purely logical procedure.}
  To gain factual knowledge, therefore, a non-logical procedure is always necessary. \ldots Though logic cannot lead us to anything new in the logical sense, it may well lead to something new in the psychological sense. Because of limitations on man's psychological abilities, the discovery of a sentence that is L-true or of a relation of L-implication is often an important cognition.
  
  But this cognition is not a factual one, and is not an insight into the state of the world; rather it is a clarification of logical relations subsisting between concepts., i.e. a clarification of relations between meanings. Suppose someone knows $\mathfrak{S}_i$ to begin with; and suppose that thereafter, by a laborious logical procedure, he finds that $\mathfrak{S}_j$ is L-implied by $\mathfrak{S}_i$. Our subject may now regard $\mathfrak{S}_j$ as known, but he may not count it as logically new: for the content of $\mathfrak{S}_j$, even though initially concealed, was from the beginning part of the content of $\mathfrak{S}_i$. Thus logical procedure, by disclosing $\mathfrak{S}_j$ and making it known, enables practical activities to be based on $\mathfrak{S}_j$. Again, two L-equivalent sentences have the same range and hence the same content; consequently they are different formulations of this common logical content. However, the psychological content (the totality of associations) of one of these sentences may be entirely different from that of the other. Carnap, \cite[p. 21-22]{carnap1958introduction}
  \end{quote}
  
  We take this understanding of proof as complete, for the purpose of comparison with student conceptualizations, which we expect, from Marton\cite{marton1976aqualitative}(is this a suitable ref?), to be partial rather than complete, and superficial, rather than deeply appreciative of the relations among the parts.
  
  Some students exhibit an understanding of proof at the "black box" level, i.e., there is understanding of the role of proof, without considering any internal structure. When a proof exists, we can know that the thing the proof proves is true, in the context that applies. We can "use that theorem". Other students, though, do not have this idea consolidated yet. For example, if we consider proof by exhaustion applied to a finite set of cardinality one, we can associate to it, the idea of a test. Students, assigned to test an algorithm for approximating the sine function, knew to invoke their implementation with the value to be tested, but did not check their result, either against the range of the sine function, or by comparison with the provided sine implementation, presenting values over 480 million.
  
  Moving to the "white box" level, we find a spectrum of variation in student understanding.
  The most opaque end of this spectrum has been called "magic incantation". In this conceptualization we find those ideas of statements that are not clear, and use of mathematical symbols that is not understood.
  "leaving it at the formal definition is kind of aaahh, I kind of work backwards with those, like I get an example, then ok this relates to this step that's what this means"
  
  Ellenberg\cite[page 409]{ellenberg2014not} reports that some mathematicians regard axioms as strings of symbols without meaning, and that this quite formal conceptualization can be contrasted with another conceptualization that axioms are true statements about logical facts. He talks of these conceptualizations being taken by the same individual at different times. 
  
  Our participants seemed to regard axioms as strings of symbols that do mean something, though that meaning the participant ascribed might not be correct (especially as participants did not always know definitions of the entities being related), or the participant might feel unable to ascribe any meaning. We did find participants who appreciated the significance of definitions. Those we found were dual majors in math.
  
  The source of such sequences served as a dimension of variation among the concepts we found in our students. Some students stressed the role of a procedure in synthesizing proofs. Proof by mathematic induction was considered preferable; a synthesis by procedure property was assigned to it. By contrast, proofs involving sequences of statements warranted by rules of inference, but otherwise unconstrained as to form, were considered less desirable. 
  
  Some students do not see the sequence of statements as carrying out a transformation process on a representation: " i get that a lot in straight programming, a lot of people have this mathematical explanations, and then in code it all of a sudden makes sense I think part of it has to do with the uh it has to do with the procedural nature of programming we're in state a, we move to state b, state c, state d and in the end we get an answer but equation is like an absolute constant total truth".
  
  Another waystation on this dimension of variation is "sequence of statements". A more elaborate idea is "sequence of statements where each next statement is justified by what when before". 
  
  Absence of attention to warrants has been reported by Alcock and Weber\cite{alcock}. Some of our students have noted this difficulty "Q: what made it difficult? A: probably not sufficiently understanding  how the logic worked i guess, for certain techniques of proofs".
  
  Another example of not finding the "connective tissue" between statements in a proof, and not noticing how a pair of statements warrants a conclusion, and not being convinced by the sequence of statements: " i'm not too fond of induction, for whatever reason, i don't know why i think that one made the least sense when i was learning  you could just say there's a base case i increment once and i guess abstracting from that, and it's true for everything it seems i don't know, it seems kind of weird, sometimes when you think about it"
  
  By contrast, some students clearly appreciate warrants: "like my (debating) points need to be clear and concise and they need to be connected one to the next. it is very much related to proofs"
  
  Some students recognized patterns in sequences of statements. Contrapositive, contradiction, categorization into cases, proof by mathematic induction have been seen as patterns, consisting of steps that can be followed. These are contrasted with what were called "logic proofs".
  
  It could be difficult to distinguish between a correct succession of logical steps from the premise(s) to the desired consequence(s) that "reaches a psychologically useful revised formulation" from "carries out a pattern". Indeed, the objectives of the course teaching proof may be met, while the preparation for the course using proof to explain the nature of, say, complexity classes, might not.
  
  
  A yet more complete concept is "finite sequence of statements, starting with the premise and ending with what we want to prove, and justified in each step." A more profound conceptualization was found "finite sequence of statements, starting with axioms and premises, proceeding by logical deduction using (valid) rules of inference to what we wanted to prove, that shows us a consequence of the definitions with which we began, an exploration in which we discover the truth value of what we wanted to show, serving after its creation as an explanation of why the theorem is true".
  
  Some students have a concept of  the exploration purpose of proof: "like everything, uh, like everything that there is to prove, it already exists. So proof is just like a way to get there."
  
  A few categories, such as those above, serve to identify a dimension of variation. When our purposes include discovering which points we may want to emphasize, we can examine the categories seeking to identify how they are related and how they differ.
  
  It can certainly be that having more categories provides more critical aspects. For example, Harel and Sowder\cite{harel1998students} offered extrinsic vs. intrinsic conviction, empirical proof schemes and their most advanced deductive proof schemes as broader categories, and seven useful subcategories of these, yielding six critical aspects that suggest what teachers could usefully vary, to help learners discern items that would advance their knowledge.
  


  
  % % %comprehension
  \newpage
  
 \section{Phenomenographic Analysis of How Students Attempt to Understand Proofs}
 
  The categories developed in the traditional phenomenographic analysis are:
   
  \begin{table}
   \caption{Categories for Student Conceptualizations of How to Understand Proofs}
   \begin{tabular}{|p{6cm}|p{6cm}|}\hline
   Category & Description\\\hline\hline
    Look up the definitions and use them (Math major) & \\\hline
     Use a diagram, visualization & \\\hline
    Go over all the logical elements from Class, related axioms and theorems & \\\hline
        Apply the Proof Pattern from Class & seen mathematic induction most often, so try that\\\hline
     
   Just Like the Examples from Class & \\\hline
  
  
  
   
   \end{tabular}
   \end{table}
   
   Ideas that would have been welcome but did not appear: 
   
    \begin{table}
     \caption{Relationships for  Student Conceptualizations of How to Understand Proofs}
     \begin{tabular}{|p{6cm}|p{6cm}|}\hline
     Idea & Description\\\hline\hline
     Notice the premises & \\\hline
     Notice the desired outcome & \\\hline
     Consider what might be deduced from the premises that might be closer to the desired outcome & \\\hline
     \end{tabular}
     \end{table}
     
     The arrangement of the categories follows that of the table, and is shown in Figure \ref{fig:HowApproach}.
     
     \begin{figure}
 \centering
 \includegraphics[width=0.7\linewidth]{./HowApproach}
 \caption{Categories from how do students approach comprehending proof}
 \label{fig:HowApproach}
 \end{figure}
 
 \begin{table}[ht]
 \caption{Critical factors for How do students approach comprehending proof?}
 \label{tab:critApproach}
 \begin{tabular}{|p{8cm}|}\hline
 Critical Aspect\\ \hline\hline
 Generalization from instance to pattern \\ \hline
 Attempt at visualization \\ \hline
 \end{tabular}
 \end{table}
 
 The first category of approach to comprehending a proof is to check whether it is one they have already examined in class.
 The next category is to check whether the proof follows a pattern that has been treated in class. The most important difference seems to be that generalization from an instance to a pattern occurs.
 
 The  next category after a pattern that has been discussed in class seems to be to engage the visual domain. It is not clear that students view the argument as a process by which one representation of a truth gets transformed into another representation, that renders the claim obvious. Thus, it is not clear what the students are attempting to visualize.
 
 
 It could be that some students are not attempting to understand proofs.
 "part of that is that there are kids in computer science who don't really want to be in CS, do the bare minimum or whatever, so i think that's part of the problem, not going to get kids who want to do proofs in cs, if they don't really want to do cs"
 
 "i will pay attention where most kids will zone out"
 
 Students can experience anxiety about mathematical notation.
" the second I see a summation, I'm like oh god this is some really long thing, or my professor's going to ask me to put in you know ask me to find the equation for this summation and it's just 'cause we did a I think it was calc2 um they had us do they gave us sequences and series and summations and they're like write the eqn for this and they were awful, just and summations are just weird because you're writing out this really long thing, used them in uh, some of the stuff we used in like 2500 for one of them we had to write out the actual summation i don't know but when you look at that little squiggly (o god)$*6$.
 
 
 Some students are attempting to understand proofs while not recognizing that they are studying a proof.
 "Were we studying proofs today?" "No" "Were we discussing certain contexts, and why certain ideas will always be true in those contexts?" "Yes" "Doesn't that seem like proof, then?" "Yes"\\
 "So, you're taking introduction to the theory of computing this semester. Do you seen any use of proofs in that course?" "No"\\
 
 
 Some students read proofs.
 
  "you just have to go through a variety of proofs a variety of contexts"
 
 Some students look up the definitions of terms used in the proofs and some do not.
 Some students think (or hope) they can solve problems involving producing proofs, without knowing the definitions of the terms used to pose the problem.
 Some students are aware that definitions are given, but "zone out" until examples are given. 
 "she'll read through the formal definition, half the class will kind of zone out, which is fine, everybody has to understand there's a formal definition but going over it personally I would use a lot of examples, i love examples" When examples are given, the students attempt to infer definitions themselves. Some students will compare the definitions they infer with the mathematical community's definitions. Some students do not.
 
 Some students think that the reading of proofs is normally conducted at the same speed as other reading, such as informal sources of information.
 
 Of students who read proofs attentively, some try to determine what rule of inference was used in moving from one statement to the next, and some do not.
 
 Some students experience transient understanding of proof techniques: "aha moment have always been proofs written for induction, despite the fact that I've done them multiple times, they go over my head and I have to relearn proofs by induction".
 
 Some students notice that lemmas can be proved and then used as building blocks in a larger argument, and some do not.
 
 Some students can identify the forms of proof learned in discrete systems, when they see them employed in proofs, such as the combination of arguing by contrapositive and modus ponens. Some cannot.
 
 Some students can identify these forms in an argument if the argument is made about concrete objects, such as cars or specific people. Some of these students have difficulty transferring this ability with concrete objects to application to abstract entities such as sets, algorithms or symbols.
 
 "i think the thing a lot of people hadn't really had to deal with before was just the level of abstraction that comes with proof writing, which is inherent with computer science, but a lot of time when we talked about problems it's always through analogies, i mean the traveling salesman problem is about cities and moving but that's not really what it's about, it's about graphs and paths"
 
 Some students who achieve with difficulty the ability to recognize the application of rules of inference in one argument about abstract entities, become quicker at recognizing arguments of similar form about other abstract entities, and some continue to achieve with difficulty, as if learning the first argument did not facilitate learning the second argument of the same form.
 
 Students would attend to diagrammatic representation of proofs, such as a block digram depicting machine descriptions packaged as input for yet other machines to process, but were not observed to employ such diagrams.
 
 Students have been seen to employ decision tree diagrams.
 
 Students would attend to algorithm representation of proofs, such as a recursive process that determines a prime factorization, but were not observed to employ such algorithmic descriptions." I wonder whether if proofs like mathematical formulations, could be rewritten as algorithms  would the computer science students find them more readily understood."
 "absolutely"
 
 "that makes a little more sense than some of the assertions, the equalities, an algorithm you can trace through, you can write it out, things like that, it's very beneficial"
 
 Students answering a list of questions, representing computer science ideas mathematically, in algorithms and in figures found the questions "interesting", "fun", "different" and "non-trivial".
 
 Except when assigned to do so, students were not observed to attempt to solve simpler problems, such as by imposing partitioning into cases. Except when assigned to do so, students were not observed to attempt to solve more general problems, as is sometimes helpful.\cite[that pin dropping probability problem]{ellenberg2014not}
 
 % % % structural relevance
 
 \newpage
 
 \section{Phenomenographic Analysis of Reasons for Teaching Proof}
 
 What is proof for? What subset of what proof is for gives us reason for teaching it?
 
   The categories developed in the traditional phenomenographic analysis are:
    
   \begin{table}
    \caption{Categories for Student Conceptualizations of Reasons for Teaching Proof}
    \begin{tabular}{|p{6cm}|p{6cm}|}\hline
    Category & Description\\\hline\hline
    
     Effective Communication of Mathematical Thoughts & \\\hline
      Understand the consequences of definition & \\\hline
      Derive mathematical formulation of intuitive ideas & \\\hline \hline
       Derive algorithms for efficiency & \\\hline 
        Tailor an algorithm so that its properties can be proven & \\\hline \hline
           Show that an algorithm meets requirements & \\\hline 
        Establish bounds on resource utilization & \\\hline 
    Understanding Algorithms and Their Properties & \\\hline
     Ensuring we know why an algorithm works &  \\\hline \hline
       Demonstrate claims (conclusively) & \\\hline 
   Distinguish the possible from the impossible & \\\hline   
     Obtain more knowledge & \\ \hline \hline 
      Find out whether hypothesis is false & \\\hline
     Increase confidence in experimental results & \\\hline 
    Do not know why & ``we do not accomplish anything''\\\hline
        Nothing desirable & \\\hline        
    Nothing of relevance & \\\hline 
    \end{tabular}
    \end{table}
    
    Ideas that would have been welcome but did not appear: 
    
     \begin{table}
      \caption{Relationships for  Student Conceptualizations of Reasons for Teaching Proof}
      \begin{tabular}{|p{6cm}|p{6cm}|}\hline
      Idea & Description\\\hline\hline
      Reasoning carefully about algorithms & \\\hline
      These reasons include being certain about algorithm properties & \\\hline
       There are good and relevant reasons for teaching proof in the computer science curriculum & \\\hline
      \end{tabular}
      \end{table}
      
      The arrangement of the categories follows that of the table, and is shown in Figure \ref{fig:ForWhat}.
      
      \begin{figure}
  \centering
  \includegraphics[width=0.7\linewidth]{./ForWhat}
  \caption{Categories from What do students think a proof is for}
  \label{fig:ForWhat}
  \end{figure}
  
   \begin{table}[ht]
   \caption{Critical factors for What do students think proof is for?}
   \label{tab:forWhat}
   \begin{tabular}{|p{8cm}|}\hline
   Critical Aspect\\ \hline\hline
    Discuss in detail occasions in which constructing proofs serves the creation of algorithms, such as pre-conditions, post-conditions, Gries-like construction \\\hline
   Sketch in more detail the domain of CSE in which students can expect to encounter proofs, also opportunities they might experience, to construct proofs     \\ \hline
    Explain purpose of proof, including starting from given, including irrefutable, demonstrate the truth value of what is to be shown, with examples from CSE  \\ \hline
   \end{tabular}
   \end{table}
   
   Because there is an unusually large number of categories in response to this research question, the categories have been grouped.
   The least sophisticated group of categories includes misunderstandings, not just incomplete, but containing misinformation. The critical aspect differentiating this group of categories from the next group is the idea that proof establishes, unequivocally, the truth value of the claim. The context of assumptions in which the claim is made is included in the understanding of the claim, in so far as that context has supplied warrants for the argument.
   
   The next group of categories are true conceptualizations that are not very detailed, such as, a proof demonstrates that a claim is true.
   The group of categories beyond that 
 
 Excerpts of student transcripts were selected on the basis of being related to this question. A dimension of variation emerged from the data, such that the excerpts seemed readily organized along this dimension.
 

 % % %application
 \newpage
 
\section{Phenomenographic Analysis of How Students Attempt to Apply Proofs (When not assigned)}

There is only one category for student responses to this question. They do not attempt proofs when not assigned.



Some students claimed they never constructed proofs when not assigned.

Code-instead  "do you ever decide on your own that you want to do a proof?
p no, I just tend I tend to just write code"

Some students did exercises related to proofs, without being assigned: "Q: Do you ever find yourself doing proofs? associated with computer science? that haven't been assigned?\\
A: That have not been assigned?\\
Q: Right, for fun, or because you want to know something?\\
A: um, he-he, well, i did find myself doing proofs, they were silly proofs, just like things about like things stuff up, yeah since i didn't have very solve it base, it was just like statements, not really just proof, just where you want to get to, so like the end result that you want to get to"
\newpage  

\section{Phenomenographic Analysis of Whether students exhibit consequences of inability (such as avoiding recursion)}

There is only one category from the traditional phenomenographic method: Students do not know when they can apply recursion. They felt they were asked to produce recursive algorithms in situations in which it applied, and that they could. They felt that such situations did not occur subsequently. Some asked their employed friends who echoed this opinion.

Some students claimed to know how to write recursive algorithms but said they never used them because they did not know when they were applicable.

% % %analysis
           
              \newpage
              
  \section{Phenomenographic Analysis of How familiar and/or comfortable are students with different (specific) proof techniques: induction, construction, contradiction?}
  
  This question was not pursued with the traditional phenomenographic method.
  
  "I'm not particularly fond of them \ldots there are different ways of strong and weak induction a whole procedure and try and so yeah there's a lot of details that go into it"
  
   "i'm probably not going to go home and do mathematical induction practice for fun"
  
   "i think like mathematical induction um uh because they verify themselves, like you  use them to check your answer, and you know if you've arrived at the expected answer most likely did it right. I generally like to do any kind of proof that has either like a given like set of steps like mathematical induction has a set of steps where you have your eventual condition, you verify your um you verify and then you do the inductive step and then you can conclude"
   
    "I'm a fan of like having like a set of steps to do something with, rather than so i know like what to do next"
  
  "because i think the thing that draws people to coding is problem solving, the feeling of achievement when something works, i mean, and so why couldn't the same thing be a part of proofs, I guess, I can see it in that way"
  
   "it's always good too because if you're even it's really reassuring when you're expecting to find something and then you find that and find out why in the process, so you have like you know what you're looking for so it's almost like working backwards, i know what i'm looking for and i know where i'm starting but if i can work both ways i can find the path pretty easily"
  
  When asked about specific proof techniques, some students mentioned proof by mathematical induction "when I start to create a proof, most commonly mathematical induction because this method of proof seems most straightforward to me, and most of these assignments we did mathematical induction, so that's what goes through my mind first."
  
  Students claimed to prefer proofs by mathematical induction on the basis that they were formulaic; supposedly, a procedure  could be used to synthesize them.
  "the thing that that induction is there are steps to it, you prove for this case, you prove for that case, plus one, I can go through those steps and by going through the steps I'm sure it's correct, because it's the right steps but in my mind it's a little shaky"

  
   "this feeling of well, I did it, but I'm not necessarily convinced"
  "yes, that's precisely the feeling I've had"
  
  When asked for specific proof techniques other than proof by mathematical induction, students knew the words contradiction and contrapositive, but sometimes could not distinguish between them.
  
"laws of logic proofs, they're a little more difficult it's almost like a puzzle"
   
"even if it's almost like in straight programming, there's a few ways to choose from, you know how to solve a certain problem, if you can choose between like one or two or three you know different steps that's fine to but when it's kind of like solve this problem here's the formal definition go for it i'm like whoa"
  
  When asked about proof by construction, some students thought this referred to construction of any proof.
  
  Some students thought proof by contradiction referred to proving the opposite of something, rather than disproving the opposite of something.
    
                
                \newpage

  \section{Phenomenographic Analysis of Which  structural elements students notice in proofs}
  
  The categories developed in the traditional phenomenographic analysis are:
      
     \begin{table}
      \caption{Categories for Student Conceptualizations of What Structural Elements are Found in Proofs}
      \begin{tabular}{|p{6cm}|p{6cm}|}\hline
      Category & Description\\\hline\hline
       Components & \\\hline
        Puzzle & \\\hline
        Pattern(s) & \\\hline
       Process Steps, State Transitions & \\\hline
       Like Programs & \\\hline
       
      \end{tabular}
      \end{table}
      
      Ideas that would have been welcome but did not appear: 
      
       \begin{table}
        \caption{Relationships for  Student Conceptualizations of What Structural Elements are Found in Proof}
        \begin{tabular}{|p{6cm}|p{6cm}|}\hline
        Idea & Description\\\hline\hline
        Good sentence structure & \\\hline
         Scoping Like Lexical Scoping & \\\hline
         
        \end{tabular}
        \end{table}
        
        The arrangement of the categories follows that of the table, and is shown in Figure \ref{fig:WhatStructure}.
        
        \begin{figure}
    \centering
    \includegraphics[width=0.7\linewidth]{./WhatStructure}
    \caption{Categories from What structure do students notice in proofs?}
    \label{fig:WhatStructure}
    \end{figure}
    
    
    Critical aspects are listed in Table \ref{tab:critWhatStruct}.
    
    \begin{table}[ht]
    \caption{Critical factors for What structure do students notice in proofs?}
    \label{tab:critWhatStruct}
    \begin{tabular}{|p{8cm}|}\hline
    Critical Aspect\\ \hline\hline
     The parts of proofs include statements, warrants, and lemmas. The idea of scope, as in proof by contradiction in which we make the temporary assumption that the consequence of an implication to be proved is false, is worth mentioning.\\ \hline
     Statements can be put in sequence. Some sequences are better than others. In particular, only sequences in which the later statements are justified by the former statements, or axioms or premises, are acceptable.\\ \hline
    \end{tabular}
    \end{table}
   
  
  Maybe for an ideal, get something from Leslie Lamport's description of using structure.
  
  Some students, in the context of hearing a presentation in an algorithms course, of a proof with a lemma, do not know, by name, what a lemma is. "What's a lemma?"
  
  Some students, in the context of planning to construct a proof, do not choose to divide and conquer the problem, breaking it into component parts, such as cases. "What good does that do, doesn't the proof become longer?"
  
  Some students describe proofs as a sequence of statements, not commenting on any structure.
  
  Some students appreciate structure: "what i had to start doing with my physics problems was breaking them down into i have this chunk, i have this chunk, i have this chunk, i'm going to label and use this chunk, i'm going to label and use this chunk and then i'm going to see how they all fit together"
  
  "very much the same logical sense, um, like with programming there's no ambiguity, everything is very structured, like proofs are structured in much the same way i enjoy programming more than regular proofs, particularly why, maybe because it's more fun to see results, when you program something"
  
  
                \newpage
    
  \section{Phenomenographic Analysis of What do students think it takes to make an argument valid?}
  
  The categories developed in the traditional phenomenographic analysis are:
    
    \begin{table}
     \caption{Categories for Student Conceptualizations of Valid Argumentation}
    \begin{tabular}{|p{6cm}|p{6cm}|}\hline
    Category & Description\\\hline\hline
    Know what's true and why & all those theorems\\\hline
    Re-use proof patterns & \\\hline
    Stick to valid rules of inference & \\\hline
    
    
    \end{tabular}
    \end{table}
    
    Ideas that would have been welcome but did not appear: 
    
     \begin{table}
      \caption{Relationships for  Student Conceptualizations of Valid Argumentation}
      \begin{tabular}{|p{6cm}|p{6cm}|}\hline
      Idea & Description\\\hline\hline
      Take note of the difference between the idea in the hypothesis, and the consequence, and consider what warranted transformations might bring the representation of the hypothesis closer to that of the consequence & \\\hline

      \end{tabular}
      \end{table}
      
      The arrangement of the categories follows that of the table, and is shown in Figure \ref{fig:Valid}.
      
      \begin{figure}
  \centering
  \includegraphics[width=0.7\linewidth]{./Valid}
  \caption{Categories from what do students think it takes to make an argument valid?}
  \label{fig:Valid}
  \end{figure}
  
      Critical aspects are listed in Table \ref{tab:critValid}.
      
      \begin{table}[ht]
      \caption{Critical factors for What structure do students think it takes to make an argument valid?}
      \label{tab:critValid}
      \begin{tabular}{|p{8cm}|}\hline
      Critical Aspect\\ \hline\hline
      Instructors do tell students about valid forms of arguments. There are some students who can recite the names of valid forms, but cannot produce arguments using them. More practice distinguishing valid arguments from invalid ones, and more practice writing arguments in valid forms might help.\\ \hline
      Students do reiterate valid proofs from class. If the assigned example matches the proof from lecture sufficiently, false positive results for understanding can occur. Students need to pay attention to how the context of definitions and the item to be proved relate to the progression of statements that demonstrates what is to be shown. Practice explicitly providing warrants might help.  \\ \hline
      \end{tabular}
      \end{table}
  
  
  Some students are not sure how to construct an argument.
  "when we hit 2100 and it was no longer like write this method, write this statement, you would then have to do this, do this, it was just a paragraph, write a stock trader that will handle this input and output this output, i panicked, i had no idea, i didn't even know, we learned how to code, but we didn't learn, we learned how to write code but we didn't learn how to code, the same we learned proofs, but we didn't learn how to write proofs, the only place we saw that was 2500 it helped close a gap for me that i, i'm still not perfect at it, it definitely helped to bring me along which was good"
  
  Some students do not understand that statements should be warranted. Without an appreciation of definitions, understanding warrants can be expected to be fraught with difficulty.  "that's where you'll find most of your problems understanding proofs, that's the first really the first class where you get it, where it's not just you know, this is the proof, take it on faith, that's what they tell you to do, but that's where you should really see why this works, let's see how this is proven, understand this as a whole, once you get it, that's where the gap is where everyone kind of loses it".
  
  Some students do not recognize a good argument when they are looking at one. "you can't expect it to be totally rigorous in decidability"
  
  Some students used confused/incorrect forms of rules of inference.
  
  Some students do not notice that the imposition of a subdivision into cases creates more premises.
  
  Some students do not notice that proof by contradiction introduces (for purposes of contradiction) a premise.

% % %synthesis
 
   
\section{Phenomenographic Analysis of Whether students incorporate structural elements in proofs}

Students have asked whether, when using categorization into cases, they must apply the same proof technique in each of the cases.
 
 Probably needs additional interviews.
 
and they you split, what do i have to do to get to that point, so you have to actually find what are the required pieces for you to solve the problem so and every single piece, they you have to prove by itself, so that can i get to the second step, can i get to the third step, because if i lose the first proof, i will never get to the second, because i already established that my second piece depends upon my first piece, so i cannot move forward so i have to divide into small pieces and try to prove them

% %evaluation
\newpage

\section{Phenomenographic Analysis of Combined Data}

This work includes two methods of analysis of combined data, the first being influenced by the traditional phenomenographic approach, so, we take each research question in turn, we take text fragments relevant to the individual research question, we obtain categories, and arrange them and infer critical aspects. Then with the arrangements and critical aspects we look for insights spanning the multiple questions. In the second method, the text fragments are not segregated by research question. Categories emerge from the whole collection of text fragments, and relationships between categories are examined using axial coding, as found in grounded theory \cite{which one uses axial?}. 

In the traditional phenomenographic analysis, in three research questions the idea of patterns appears. The ideas of process steps and of visualization also feature prominently. Some students seem to be attempting to understand the structure and validity of proofs by using their knowledge of patterns, process steps and visualization.
This suggests that explicit use of patterns, process steps and visualization, comparing and contrasting these existing ideas with ideas from proof will help students learn the new ideas.


definitions vs examples, examples are easier, value of definitions not necessarily appreciated.

Use of examples implies hope that generalization will occur.

Recognition that generalization is difficult.