Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
responded to edits and final read-aloud fix
  • Loading branch information
rcc02007 committed Apr 3, 2020
1 parent f298d83 commit dcdedb5
Showing 1 changed file with 48 additions and 50 deletions.
98 changes: 48 additions & 50 deletions ASEE-DELOS_Cooper.tex
Expand Up @@ -181,7 +181,7 @@ with the same data set and submit reports graded with the rubric in
Appendix A. Lab \#1 asks students to quantify differences in machining
methods between band saw and computer numerical control (CNC) parts. Labs \#2-4
ask students to quantify differences between rational predictions using
analytical and numerical models and empirical measurements for static and
rational models and empirical measurements for static and
dynamic cantilever beams. In the PjBL activity, the Lab \#5 competition, the
students are given the task to create a design of experiments, create a
predictive model, and use engineering judgment to measure the mass of an object
Expand Down Expand Up @@ -229,7 +229,7 @@ experiments, take measurements, and create finite element analysis models. The
competition does not have calibration weights, so the students have to rely on
rational predictions and engineering judgments. The
competition ends with the submission of their best estimate of object mass with
a propagation of error and the Methods section. The lab group with the most
a propagation of error and the lab report's Methods section. The lab group with the most
accurate measurement is awarded a \$150-prize. After the prize is awarded, the
actual object masses are announced. The lab groups use week 12 to revise
their approach and submit the lab report. The goal is to encourage students to
Expand All @@ -247,34 +247,33 @@ clear feedback on the final error in the predicted results.
%--------------------------------------
%report 23.7442 4.0000 1776.0000 0.0000
%======================================
The course focuses on problem-solving and technical writing. The F-value in a
one-way repeated Analysis of Variance, using the Python package
statsmodels\cite{seabold2010} was 23.74 between labs 0-4 with 445
students indicating that there was a statistically significant affect on lab
report grades. In Fig.~\ref{quality}(a), the scores of each lab group is fit to
a linear model to determine the change in report grade per report between Labs
\#0-4. The goal was to have the entire class in the green ``continuous
improvement''-area. In Fall~2018, 56\% of the class continually improved and in
Fall~2019, 59\% of the class continually improved their scores. The ``maintain
quality'' area represents students that write reports of high quality initially,
but do not improve during the course of the class. In Fall~2018 and Fall 2019,
the students that maintained quality accounted for 43\% and 36\%, respectively.
The remaining 1\% and 4\% of the class did not improve or meet specifications
for lab reports, in Fall 2018 and 2019, respectively. The grades from Labs~\#5-6
are shown in Fig.~\ref{quality}(b). Lab~\#5 was the PjBL contest and marked a
significant increase in expectations. The results of this study, suggest that
students are able to incorporate feedback from teaching assistants and myself
and show improvements in technical writing. The Labs increased in difficulty, so
even the groups of students that maintained their grade at the specified level
show marked improvement in communicating difficult concepts.

Regarding the effectiveness of specifications grading in technical writing,
there is still a normal distribution of grades with the class mean between 80
and 85~points and grades increased throughout the semester. One argument against

The course focuses on problem-solving and technical writing. In Fig.~\ref{quality}(a),
the scores of each lab group is fit to a linear model to measure average increase in
grade per report between Labs \#0-4. The goal was to have the entire class in the green
``continuous improvement''-area. In Fall~2018, 56\% of the class continually improved and
in Fall~2019, 59\% of the class continually improved their scores. The ``maintain
quality'' area represents students that write reports of high quality initially, but do
not improve during the course of the class. In Fall~2018 and Fall 2019, the students that
maintained quality accounted for 43\% and 36\%, respectively. The remaining 1\% and 4\%
of the class did not improve or meet specifications for lab reports, in Fall 2018 and
2019, respectively. The F-value in a one-way repeated Analysis of Variance of lab report
scores, using the Python package statsmodels\cite{seabold2010}, was 23.74 between labs 0-4
with 445 students indicating that there was a statistically significant affect on lab
report grades. The grades from Labs~\#5-6 are shown in Fig.~\ref{quality}(b). Lab~\#5 was
the PjBL contest and marked a significant increase in expectations. The results of this
study, suggest that students are able to incorporate feedback from teaching assistants and
myself and show improvements in technical writing. The Labs increased in difficulty, so
even the groups of students that maintained their grade at the specified level show marked
improvement in communicating difficult concepts.

I found specifications grading in technical writing to be an effective method of
evaluation. The grades are normally distributed with the class mean increasing from 80
to 85~points. One argument against
specifications grading is that students may not be motivated to increase their
grade because once the grade is above passing there is no incentive to improve.
I find a clear increase in grades throughout the semester, and the students
that were in the ``maintaining poor quality'' regime did fail and redo lab reports.
that were in the ``maintain poor quality'' regime did fail and redo lab reports.
The students that did not improve found great difficulty in Labs~\#5-6, most
failing those assignments and revising their work. The specifications grading
also has the most noticeable effect on under-performing students. The students
Expand Down Expand Up @@ -321,7 +320,7 @@ progress was sustained and labs did not become more demanding.
semester, Green indicates students that passed Report~\#0 whose scores continued
to increase throughout the semester, and orange are students that passed
Report~\#0, but their scores decreased throughout the semester. The orange marks
in the red sections, "maintain poor quality" were at risk of failing other lab
in the red sections, ``maintain poor quality'' were at risk of failing other lab
reports. In (b), box plots of the scores from 2018 and 2019 on reports 0-6 are
plotted. The median is shown by a horizontal line, the notches indicate the
confidence interval, the whiskers denote the range of scores, with outliers
Expand All @@ -335,31 +334,30 @@ histogram of errors based upon reported results demonstrate the range of
effectiveness of each lab group's experimental work. In Fall~2018 and Fall~2019,
the average and standard deviation in error to measure a 32-g object was
18.3$\pm$32.8~g and 11.4$\pm$26.7~g, respectively. While top three most accurate
reports had errors less than 4\%. The competition provides very specific
feedback to lab groups, and provides a non-grade-based metric to evaluate
reports had errors less than 4\%. The competition provides specific
feedback to lab groups, and a non-grade-based metric to evaluate
student effort and learning.

This PjBL Lab qualitatively had the highest enthusiasm and participation from
the students. Student SET responses included, ``I liked the mass measuring
contest!'', ``I liked using ANSYS and the competition.'', ``I liked the
competition where the answer was unknown. I think that was the most beneficial
thing we did and I think more of those labs would be helpful.'' Attendance to
announce winners of the contest was not mandatory, but over 90\% of the class was
present. Students compared answers, studied methods, and results. After the
object masses were given to the class, they revised their methods one more time
to reduce errors in their data collection and processing. The benefit of the
contest was the increased enthusiasm in studying beam dynamics and finite
element methods. Even students that had very high errors, had finite element
models with demonstrated convergence, fast fourier transform analysis of natural
frequencies of cantilever beams. These competitions work best when the learning
happens whether or not the group wins\cite{burguillo2010}.
This PjBL Lab qualitatively had the highest enthusiasm and participation from the
students. Student SET responses included, ``I liked the mass measuring contest!'', ``I
liked using ANSYS and the competition.'', ``I liked the competition where the answer was
unknown. I think that was the most beneficial thing we did and I think more of those labs
would be helpful.'' Attendance to announce winners of the contest was not mandatory, but
over 90\% of the class was present. Students compared answers, studied methods, and
results. After the object masses were given to the class, they revised their methods one
more time to reduce errors in their data collection and processing. These competitions
work best when the learning happens whether or not the group wins\cite{burguillo2010}. The
benefit of the contest was the increased enthusiasm in studying beam dynamics and finite
element methods. Even students that had very high errors demonstrated finite element
models convergence and fast fourier transform analysis of natural frequencies of
cantilever beams.

\begin{figure}[ht!]
\centering
\includegraphics[width=5in]{./track_progress/mass_measure.png}
\caption{Plotted above is a histogram of the reported errors from Fall~2018
and Fall~2019 for the mass measurement contest. The average mass reported in
Fall~2018 and Fall~2019 was 18~$\pm$~33~g and 41~$\pm$~27~g, respectively with
and Fall~2019 for the mass measurement contest. The average error in mass reported in
Fall~2018 and Fall~2019 was 18~$\pm$~33~g and 11.4~$\pm$~27~g, respectively with
error reported as standard deviation. The actual mass measurements were
32~$\pm$~2~g. The histogram is the error=(reported value - the actual value). \label{contest}}
\end{figure}
Expand Down Expand Up @@ -409,18 +407,18 @@ over 50\% of students increased technical writing quality. Access to interactive
notebooks increased the variety and use of the lab handouts. Using Jupyter
notebook handouts created a medium that mixed background information, data
processing, and simple engineering models. The Jupyter notebooks helped to close
the gap between rational, thinking design and empirical, hands-on design. The
project-based upper engineering lab course redesign has been a success. Using
the gap between rational and empirical, hands-on design. The
project-based upper-level engineering lab course redesign has been a success. Using
the 2019-2020 senior capstone students, I found a statistically significant
increase in preparation for engineering design from taking the lab course with
PjBL.

Some ongoing work will be to evaluate the effectiveness of individual changes in
the course. Specifications grading is a novel way to asses engineering students'
technical writing skills. I believe the process of revising reports provided
technical writing skills. I believe the process of revising reports provides
much-needed practice for students, but it would be interesting to see what
fraction of the class has measurable increase in writing quality without this
process. I assume the PjBL competition was a big motivational and preparational
process. I assume the PjBL competition is a big motivational and preparational
tool, but there may be other sources of motivation and preparation. Some future
work is to compare results between a competition-based PjBL and
PjBL component with no competition and to incorporate senior design grades into
Expand Down

0 comments on commit dcdedb5

Please sign in to comment.