tutorial-vignette.RmdThis tutorial demonstrates how to conduct an Ordered Network Analysis
using the ona R package. If you are new to ONA as an
analytic technique, Tan et al. (2022) provides a more detailed
explication of its theoretical and methodological foundations.
Because ONA shares some conceptual and procedural similarities with Epistemic Network Analysis (ENA), you may also want to read the ENA tutorial paper, ENA example paper, and ENA mathematical foundation paper to familiarize yourself with the theoretical and methodological foundations of ENA.
For a better reading experience, I suggest you use
visual mode instead of source mode to read and
use this rmd file. Switch mode by clicking on the Source or Visual on
the left top corner.
Due to the practical focus of this tutorial, I omitted detailed explanations of the theoretical, methodological, and mathematical foundations of ONA that are crucial for informed, theory-based learning analytics research. Please reach out to Yuanru Tan (yuanru.tan@wisc.edu) with any questions! Also, welcome and appreciate any feedback that you might have to make this tutorial more helpful.
Note: when installing ona or tma package, if you are a Mac user and you run into “package has non-zero exit status” after replying “Yes” to the question “do you want to attempt to install these from sources? (Yes/no/cancel)”, please follow this link to install the gfortran that matches the version of your MacOS: https://github.com/fxcoudert/gfortran-for-macOS/releases. After you finish the gfortran process, restart your R studio before you install the packages again.
Install the ona package and load the ona library after installing.
install.packages("ona", repos = c("https://cran.qe-libs.org"))
#>
#> The downloaded binary packages are in
#> /var/folders/5l/hqcrmb1n08b082ycb_ybqlnc0000gp/T//Rtmph5YdLx/downloaded_packages
library(ona)Then, install the other package that is required for ONA analysis.
install.packages("tma", repos = c("https://cran.qe-libs.org"))
#>
#> The downloaded binary packages are in
#> /var/folders/5l/hqcrmb1n08b082ycb_ybqlnc0000gp/T//Rtmph5YdLx/downloaded_packages
library(tma )The dataset we will use as an example, RS.data, is
included in the ONA package. Note that the
RS.data file in the package is only a subset of the full
dataset, and is thus intended for demonstration purposes only.
To start, pass RS.data from the ONA package to a data
frame named data.
data = ona::RS.dataUse the head() function in R to subset and preview the first six rows present in the input data frame to familiarize yourself with the data structure.
head(data)
#> UserName Condition CONFIDENCE.Pre CONFIDENCE.Post CONFIDENCE.Change
#> 1 steven z FirstGame 7 8 1
#> 2 akash v FirstGame 6 8 2
#> 3 alexander b FirstGame 5 7 1
#> 4 brandon l FirstGame 5 6 1
#> 5 brandon l FirstGame 5 6 1
#> 6 christian x FirstGame 4 4 0
#> C.Level.Pre NewC.Change C.Change Timestamp ActivityNumber GroupName
#> 1 High.Pre Pos.Change Pos.Change 9/17/2013 9:43 1 Electric
#> 2 High.Pre Pos.Change Pos.Change 9/17/2013 9:44 1 Electric
#> 3 Low.Pre Pos.Change Pos.Change 9/17/2013 9:46 1 Electric
#> 4 Low.Pre Pos.Change Pos.Change 9/17/2013 9:58 1 Electric
#> 5 Low.Pre Pos.Change Pos.Change 9/17/2013 9:58 1 Electric
#> 6 Low.Pre No.Change Neg.Change 9/19/2013 9:40 1 Electric
#> GameHalf GameDay
#> 1 First 1
#> 2 First 1
#> 3 First 1
#> 4 First 1
#> 5 First 1
#> 6 First 3
#> text Data
#> 1 Steven 0
#> 2 Hey, I am Akash 0
#> 3 I'm Alex 0
#> 4 OK. 0
#> 5 OK. 0
#> 6 @Justin Kim: Should we have received an email about the bio pages? 0
#> Technical.Constraints Performance.Parameters Client.and.Consultant.Requests
#> 1 0 0 0
#> 2 0 0 0
#> 3 0 0 0
#> 4 0 0 0
#> 5 0 0 0
#> 6 0 0 0
#> Design.Reasoning Collaboration Context
#> 1 0 0 Public
#> 2 0 0 Public
#> 3 0 0 Public
#> 4 0 0 Public
#> 5 0 0 Public
#> 6 0 0 PublicRS.data consists of discourse from RescuShell,
an online learning simulation where students work as interns at a
fictitious company to solve a realistic engineering design problem in a
simulated work environment. Throughout the internship, students
communicate with their project teams and mentors via online chat, and
these chats are recorded in the “text” column. A set of qualitative
codes were applied to the data in the “text” column, where a value of 0
indicates the absence of the code and a value of 1 indicates the
presence of the code in a given line.
To construct an ONA model, identify which columns in the data to use for the parameters required by the ONA modeling function.
In ONA, units can be individuals, ideas, organizations, or any other entity whose structure of connections you want to model. To set the units parameter, specify which column(s) in the data contain the variables that identify unique units.
For this example, choose the “Condition” column and the “UserName” column to define the units. The “Condition” column has two unique values: FirstGame, and SecondGame, representing novice users and relative expert users, respectively, as some students participated in RescuShell after having already completed a different engineering virtual internship. The “UserName” column includes unique user names for all students (n=48). This way of defining the units means that ONA will construct a network for each student in each condition.
my_units <- c("Condition", "UserName") Next, specify the columns that contain the codes. Codes are
concepts whose pattern of association you want to model for each unit.
ONA represent codes as nodes in the networks and co-occurrences of codes
as edges. Most researchers use binary coding in ONA analyses, where the
values in the code columns are either 0 (indicating that the code is not
present in that line) or 1 (indicating that the code is present in that
line). RS.data contains six code columns, all of which will
be used here.
my_codes = c(
'Data',
'Technical.Constraints',
'Performance.Parameters',
'Client.and.Consultant.Requests',
'Design.Reasoning',
'Collaboration')The parameter to specify conversations in ONA is called “my_hoo_rules”, where “hoo” is an abbreviation of “horizon of observation.”
The conversation parameter determines which lines in the data can be connected. Codes in lines that are not in the same conversation cannot be connected. For example, you may want to model connections within different time segments, such as days, or different steps in a process, such as activities.
In our example, choose the “Condition”, “GroupName”, and “ActivityNumber” columns to define the conversations. These choices indicate that connections can only happen between students who were in the same condition (FirstGame or SecondGame) and on the same project team (group), and within the same activity. This definition of conversation reflects what actually happened in the simulation: in a given condition, students only interacted with those who were in the same group, and each activity occurred on a different day.
Choose the combination of “Condition” column, “GroupName” column, and “ActivityNumber” column to define the conversation parameter.
my_hoo_rules <- conversation_rules(
(Condition %in% UNIT$Condition &
GroupName %in% UNIT$GroupName &
ActivityNumber %in% UNIT$ActivityNumber))Once the conversation parameter is specified, a window method needs to be specified. Whereas the conversation parameter specifies which lines can be related, the window parameter determines which lines within the same conversation are related. The most common window method used in ONA is called a moving stanza window, which is what will be used here.
Briefly, a moving stanza window is a sliding window of fixed length that moves through a conversation to detect and accumulate code co-occurrences in recent temporal context. The lines within a designated stanza window are considered related to each other. For instance, if the moving stanza window is 7, then each line in the conversation is linked to the six preceding lines. See Siebert-Evenstone et al. (2017) and Ruis et al. (2019) for more detailed explanations of windows.
Specify a moving stanza window size by passing a numerical value to
the window_size parameter.
window_size = 7To specify an infinite stanza window in ONA, set the size of the
moving window equal to or larger than the number of lines in the longest
conversation. For example, set window_size = 4000, which is
greater than the total number of rows in our dataset (nrows=3,824). The
infinite stanza window works the same way as a moving stanza window, but
there is no limit on the number of previous lines that are included in
the window besides the conversation itself.
Metadata columns can be included if desired. Metadata columns are not required to construct an ONA model, but they provide information that can be used to subset units in the resulting model.
metaCols = c("CONFIDENCE.Change","CONFIDENCE.Pre","CONFIDENCE.Post","C.Change")Now that all the parameters are specified, connections can be accumulated. For each unit, the ONA algorithm uses a moving stanza window to identify connections formed from a current line of data (e.g., a turn of talk), or response, to the preceding lines within the window (the common ground).
Unlike in unordered model such as ENA, where connections among codes are recorded in a symmetric adjacency matrix, ONA accounts for the order in which the connections occur by constructing an asymmetric adjacency matrix for each unit; that is, the number of connections from code A to code B may be different than the number of connections from B to A.
To accumulate connections, pass the parameters specified to the
contexts and accumulate_contexts functions,
and store the output in an object (in this case,
accum.ona).
accum.ona <-
contexts(data,
units_by = my_units,
hoo_rules = my_hoo_rules) |>
accumulate_contexts(codes = my_codes,
decay.function = decay(simple_window, window_size = 7),
meta.data = metaCols,
return.ena.set = FALSE) # keep this as FALSE to get an ONA model, otherwise it will return an undirected model)After accumulation, call the model function to construct
an ONA model. ONA currently implements singular value
decomposition (SVD) and means rotation (MR) to perform
dimensional reduction.
To create an ONA model using SVD, pass the accum.ona
object to the model function.
set.ona <-
model(accum.ona)
#> Registered S3 method overwritten by 'rENA':
#> method from
#> plot.ena.set onaWhen there are two discrete groups to compare, a means rotation can be used.
A means rotation is specified using rotate.using ="mean"
in the model function. Additionally, the model
function expects rotation.params to be a list
with two named elements, each containing a logical vector representing
the rows of units to be included in each group.
Here, construct the ONA model as shown below.
Information about an ONA model is now stored in the R object
set.ona. Users can explore the data stored in the object by
typing set.ona$ and select items from the drop down list.
Here, we briefly explain the top-level items in
set.ona$.
Because ONA accounts for the order in which the connections occur by constructing an asymmetric adjacency matrix for each unit, connection counts from code A to code B and from B to A, as well as self-connections for each code (from A to A) are recorded. Thus, because six codes were included in the model, the cumulative adjacency vector for each unit contains 36 terms (n^2).
head(set.ona$connection.counts,3)
#> Condition UserName ENA_UNIT Data & Data
#> <ena.metadata> <ena.metadata> <ena.metadata> <ena.co.occurrence>
#> 1: FirstGame steven z FirstGame::steven z 26
#> 2: FirstGame akash v FirstGame::akash v 72
#> 3: FirstGame alexander b FirstGame::alexander b 11
#> Technical.Constraints & Data Performance.Parameters & Data
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 44.0 32
#> 2: 102.5 66
#> 3: 21.5 15
#> Client.and.Consultant.Requests & Data Design.Reasoning & Data
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 12 52
#> 2: 10 88
#> 3: 4 14
#> Collaboration & Data Data & Technical.Constraints
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 8 27.0
#> 2: 12 106.5
#> 3: 1 11.5
#> Technical.Constraints & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 63
#> 2: 193
#> 3: 40
#> Performance.Parameters & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 29.5
#> 2: 90.5
#> 3: 15.5
#> Client.and.Consultant.Requests & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 8.5
#> 2: 23.5
#> 3: 1.0
#> Design.Reasoning & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 62.0
#> 2: 160.5
#> 3: 23.0
#> Collaboration & Technical.Constraints Data & Performance.Parameters
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 11.0 24
#> 2: 29.0 77
#> 3: 15.5 10
#> Technical.Constraints & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 35.5
#> 2: 91.5
#> 3: 17.5
#> Performance.Parameters & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 34
#> 2: 72
#> 3: 10
#> Client.and.Consultant.Requests & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 5.5
#> 2: 20.5
#> 3: 3.0
#> Design.Reasoning & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 42
#> 2: 77
#> 3: 14
#> Collaboration & Performance.Parameters Data & Client.and.Consultant.Requests
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 7.5 6
#> 2: 14.5 18
#> 3: 1.0 5
#> Technical.Constraints & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 7.5
#> 2: 19.5
#> 3: 7.0
#> Performance.Parameters & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 11.5
#> 2: 18.5
#> 3: 3.0
#> Client.and.Consultant.Requests & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 5
#> 2: 12
#> 3: 1
#> Design.Reasoning & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 15.5
#> 2: 9.0
#> 3: 3.0
#> Collaboration & Client.and.Consultant.Requests Data & Design.Reasoning
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 2 19
#> 2: 3 86
#> 3: 0 10
#> Technical.Constraints & Design.Reasoning
#> <ena.co.occurrence>
#> 1: 50.0
#> 2: 152.5
#> 3: 17.0
#> Performance.Parameters & Design.Reasoning
#> <ena.co.occurrence>
#> 1: 20
#> 2: 69
#> 3: 9
#> Client.and.Consultant.Requests & Design.Reasoning
#> <ena.co.occurrence>
#> 1: 5.5
#> 2: 16.0
#> 3: 2.0
#> Design.Reasoning & Design.Reasoning Collaboration & Design.Reasoning
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 59 6.0
#> 2: 136 33.5
#> 3: 19 4.0
#> Data & Collaboration Technical.Constraints & Collaboration
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0 7.0
#> 2: 15 27.0
#> 3: 6 18.5
#> Performance.Parameters & Collaboration
#> <ena.co.occurrence>
#> 1: 0.5
#> 2: 8.5
#> 3: 6.0
#> Client.and.Consultant.Requests & Collaboration
#> <ena.co.occurrence>
#> 1: 0
#> 2: 2
#> 3: 0
#> Design.Reasoning & Collaboration Collaboration & Collaboration
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 5.0 0
#> 2: 42.5 14
#> 3: 7.0 9In set.ona$connection.counts, the value for each unique
co-occurrence of codes is an integer equal or greater than 0, because
they represent the directional connection counts between each pair of
codes. In set.ona$line.weights, the connection counts are
sphere normalized, and so the values are between 0 and 1.
head(set.ona$line.weights,3)
#> Condition UserName ENA_UNIT ENA_DIRECTION
#> <ena.metadata> <ena.metadata> <ena.metadata> <ena.metadata>
#> 1: FirstGame steven z FirstGame::steven z response
#> 2: FirstGame akash v FirstGame::akash v response
#> 3: FirstGame alexander b FirstGame::alexander b response
#> Data & Data Technical.Constraints & Data
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.1543564 0.2612185
#> 2: 0.1619657 0.2305762
#> 3: 0.1424314 0.2783886
#> Performance.Parameters & Data Client.and.Consultant.Requests & Data
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.1899771 0.07124140
#> 2: 0.1484686 0.02249524
#> 3: 0.1942246 0.05179323
#> Design.Reasoning & Data Collaboration & Data Data & Technical.Constraints
#> <ena.co.occurrence> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.3087127 0.04749427 0.1602931
#> 2: 0.1979581 0.02699429 0.2395743
#> 3: 0.1812763 0.01294831 0.1489055
#> Technical.Constraints & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 0.3740173
#> 2: 0.4341581
#> 3: 0.5179323
#> Performance.Parameters & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 0.1751351
#> 2: 0.2035819
#> 3: 0.2006988
#> Client.and.Consultant.Requests & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 0.05046266
#> 2: 0.05286381
#> 3: 0.01294831
#> Design.Reasoning & Technical.Constraints
#> <ena.co.occurrence>
#> 1: 0.3680806
#> 2: 0.3610486
#> 3: 0.2978111
#> Collaboration & Technical.Constraints Data & Performance.Parameters
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.06530462 0.1424828
#> 2: 0.06523619 0.1732133
#> 3: 0.20069875 0.1294831
#> Technical.Constraints & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 0.2107558
#> 2: 0.2058314
#> 3: 0.2265954
#> Performance.Parameters & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 0.2018506
#> 2: 0.1619657
#> 3: 0.1294831
#> Client.and.Consultant.Requests & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 0.03265231
#> 2: 0.04611524
#> 3: 0.03884492
#> Design.Reasoning & Performance.Parameters
#> <ena.co.occurrence>
#> 1: 0.2493449
#> 2: 0.1732133
#> 3: 0.1812763
#> Collaboration & Performance.Parameters Data & Client.and.Consultant.Requests
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.04452587 0.03562070
#> 2: 0.03261810 0.04049143
#> 3: 0.01294831 0.06474153
#> Technical.Constraints & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 0.04452587
#> 2: 0.04386571
#> 3: 0.09063815
#> Performance.Parameters & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 0.06827301
#> 2: 0.04161619
#> 3: 0.03884492
#> Client.and.Consultant.Requests & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 0.02968392
#> 2: 0.02699429
#> 3: 0.01294831
#> Design.Reasoning & Client.and.Consultant.Requests
#> <ena.co.occurrence>
#> 1: 0.09202014
#> 2: 0.02024571
#> 3: 0.03884492
#> Collaboration & Client.and.Consultant.Requests Data & Design.Reasoning
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.011873567 0.1127989
#> 2: 0.006748571 0.1934590
#> 3: 0.000000000 0.1294831
#> Technical.Constraints & Design.Reasoning
#> <ena.co.occurrence>
#> 1: 0.2968392
#> 2: 0.3430524
#> 3: 0.2201212
#> Performance.Parameters & Design.Reasoning
#> <ena.co.occurrence>
#> 1: 0.1187357
#> 2: 0.1552171
#> 3: 0.1165348
#> Client.and.Consultant.Requests & Design.Reasoning
#> <ena.co.occurrence>
#> 1: 0.03265231
#> 2: 0.03599238
#> 3: 0.02589661
#> Design.Reasoning & Design.Reasoning Collaboration & Design.Reasoning
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.3502702 0.03562070
#> 2: 0.3059352 0.07535905
#> 3: 0.2460178 0.05179323
#> Data & Collaboration Technical.Constraints & Collaboration
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.00000000 0.04155748
#> 2: 0.03374286 0.06073714
#> 3: 0.07768984 0.23954367
#> Performance.Parameters & Collaboration
#> <ena.co.occurrence>
#> 1: 0.002968392
#> 2: 0.019120952
#> 3: 0.077689840
#> Client.and.Consultant.Requests & Collaboration
#> <ena.co.occurrence>
#> 1: 0.000000000
#> 2: 0.004499048
#> 3: 0.000000000
#> Design.Reasoning & Collaboration Collaboration & Collaboration
#> <ena.co.occurrence> <ena.co.occurrence>
#> 1: 0.02968392 0.00000000
#> 2: 0.09560476 0.03149333
#> 3: 0.09063815 0.11653476For each unit, ONA produces an ONA point in a two-dimensional space formed by the first two dimensions of the dimensional reduction.
Here, the MR1 column represents the x-axis coordinate for each unit, and the SVD2 column represents the y-axis coordinate for each unit.
head(set.ona$points,3)
#> Condition UserName ENA_UNIT ENA_DIRECTION
#> <ena.metadata> <ena.metadata> <ena.metadata> <ena.metadata>
#> 1: FirstGame steven z FirstGame::steven z response
#> 2: FirstGame akash v FirstGame::akash v response
#> 3: FirstGame alexander b FirstGame::alexander b response
#> MR1 SVD2 SVD3 SVD4
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.00753635 -0.05350532 0.02308722 0.03365899
#> 2: -0.07719283 0.01840812 -0.01485049 -0.03634380
#> 3: -0.20600855 -0.05806135 -0.08409658 0.13340227
#> SVD5 SVD6 SVD7 SVD8
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.18576251 -0.064647347 -0.016387339 0.02504138
#> 2: 0.02065882 -0.003406081 -0.008026705 -0.01584017
#> 3: 0.03017168 -0.046102014 -0.095674649 -0.10337234
#> SVD9 SVD10 SVD11 SVD12
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: -0.04662639 0.01654204 -0.0050339193 0.001089911
#> 2: -0.01879391 0.03338419 -0.0082095228 -0.004596587
#> 3: 0.14247946 0.01749875 0.0005982879 -0.073284812
#> SVD13 SVD14 SVD15 SVD16
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.02015019 -0.03170014 0.014028551 -0.03292583
#> 2: 0.05034682 -0.02280942 0.042727200 0.02373320
#> 3: -0.01959735 -0.01633710 -0.002697103 -0.03717024
#> SVD17 SVD18 SVD19 SVD20
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: -0.01286482 0.0002353795 -0.014261325 0.01159296
#> 2: -0.02829064 -0.0208970211 0.010120401 -0.01752303
#> 3: 0.02019079 -0.0119060565 -0.006518936 -0.03746771
#> SVD21 SVD22 SVD23 SVD24
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.0009366777 -0.033851942 -0.003230414 -0.016553686
#> 2: 0.0045814224 -0.006533689 -0.007668149 0.017903544
#> 3: 0.0075254949 0.038768078 0.008012232 0.009589488
#> SVD25 SVD26 SVD27 SVD28
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 2.124235e-03 0.008351416 0.024600338 -0.0003563738
#> 2: -4.643724e-05 -0.013146155 -0.003761067 -0.0172067645
#> 3: 1.347210e-03 0.014229380 0.006423058 0.0020076670
#> SVD29 SVD30 SVD31 SVD32
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.006848068 0.009214306 0.000689622 0.001926662
#> 2: -0.009072690 -0.006574454 0.001518995 0.006561024
#> 3: 0.003970270 -0.003266165 0.004738366 0.003057179
#> SVD33 SVD34 SVD35 SVD36
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: -0.0001196152 0.0057286766 0.003116433 0.003361801
#> 2: 0.0037200464 0.0019185441 0.006447044 -0.001230173
#> 3: 0.0018318401 0.0008428842 -0.004884616 0.001080194The rotation matrix used during the dimensional reduction can be
accessed through set.ona$rotation. This is mostly useful
when you want to construct an ONA metric space using one dataset and
then project ONA points from different data into that space, as in
section 8.1.
head(set.ona$rotation.matrix,3)
#> codes MR1 SVD2
#> <ena.metadata> <ena.dimension> <ena.dimension>
#> 1: Data & Data -0.06930733 -0.4261566
#> 2: Technical.Constraints & Data -0.18483941 -0.3305414
#> 3: Performance.Parameters & Data 0.16511484 -0.4098705
#> SVD3 SVD4 SVD5 SVD6
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.041261221 -0.24813254 -0.12386178 -0.1212851
#> 2: -0.001705309 -0.09652822 0.29776415 -0.2487902
#> 3: -0.057056137 -0.06943836 0.02894105 -0.1666548
#> SVD7 SVD8 SVD9 SVD10
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: -0.1813800 -0.02634818 0.02463562 -0.1685089
#> 2: 0.4530700 -0.08131861 0.19027807 -0.2985731
#> 3: -0.1512543 0.26221091 0.15918103 0.2634069
#> SVD11 SVD12 SVD13 SVD14
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.02779782 0.1634913 0.03515259 -0.1307011
#> 2: 0.03356852 -0.1788196 0.24614774 -0.1162268
#> 3: 0.30265567 0.0364710 -0.16880948 0.2246206
#> SVD15 SVD16 SVD17 SVD18
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.11108864 0.1678676 -0.21955326 0.4390146
#> 2: -0.06526278 0.2427948 0.31464439 -0.1658976
#> 3: 0.04608908 -0.1521164 -0.06523085 -0.3436932
#> SVD19 SVD20 SVD21 SVD22
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.263975884 -0.03791243 -0.26637411 -0.16378180
#> 2: -0.023955700 -0.04756904 0.05450570 0.09542229
#> 3: 0.005934216 -0.21114098 0.07531336 0.18550364
#> SVD23 SVD24 SVD25 SVD26
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: -0.04863544 -0.03046897 0.01129295 0.07990007
#> 2: 0.04913771 0.09391774 -0.07213795 -0.01391693
#> 3: -0.20153993 0.11835360 -0.08759355 -0.06613629
#> SVD27 SVD28 SVD29 SVD30
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: -0.10906748 0.1218933 0.05135484 -0.08227500
#> 2: -0.03060512 -0.0431860 -0.04667436 -0.03855961
#> 3: 0.01739627 0.1294605 0.01342884 -0.17369350
#> SVD31 SVD32 SVD33 SVD34
#> <ena.dimension> <ena.dimension> <ena.dimension> <ena.dimension>
#> 1: 0.09675121 -0.205485302 -0.159446843 -0.12907089
#> 2: -0.06592640 -0.006227139 -0.006191610 0.02971639
#> 3: 0.07209455 -0.153340746 -0.009824955 0.17078477
#> SVD35 SVD36
#> <ena.dimension> <ena.dimension>
#> 1: 0.14685408 0.01703190
#> 2: 0.11327562 0.07985307
#> 3: -0.02710075 0.06842069set.ona$meta.data gives a data frame that includes all
the columns except for the code connection columns.
head(set.ona$meta.data,3)
#> Condition UserName ENA_UNIT
#> <ena.metadata> <ena.metadata> <ena.metadata>
#> 1: FirstGame steven z FirstGame::steven z
#> 2: FirstGame akash v FirstGame::akash v
#> 3: FirstGame alexander b FirstGame::alexander bOnce an ONA model is constructed, ONA networks can be visualize. The
plotting function in ONA is called plot.
Before plotting, you can set up several global parameters to ensure consistency across plots. These parameters will be clearer in subsequent sections.
node_size_multiplier = 0.4 # scale up or down node sizes
node_position_multiplier = 1 # zoom in or out node positions
point_position_multiplier = 1.5 # zoom in or out the point positions
edge_arrow_saturation_multiplier = 1.5 # adjust the chevron color lighter or darker
edge_size_multiplier = 1 # scale up or down edge sizesMean ONA networks can be plotted for each of the conditions along with their subtracted network.
First, plot the mean network for the FirstGame condition. Use a pipe
|> to connect the edges function and the
nodes function. Users are only required to specify the
weights parameter, as the remaining parameters have default
values unless specified otherwise.
Use set.ona$points to subset the rows that are in each
condition and plot the units in each condition as a different color.
The points are specified in the units function. The
edges and nodes functions remain the same as
above.
plot(set.ona, title = "FirstGame (red) mean network") |>
units(
points=set.ona$points$Condition$FirstGame,
point_position_multiplier = point_position_multiplier,
points_color = c("red"),
show_mean = TRUE, show_points = TRUE, with_ci = TRUE) |>
edges(
weights =set.ona$line.weights$Condition$FirstGame,
edge_size_multiplier = edge_size_multiplier,
edge_arrow_saturation_multiplier = edge_arrow_saturation_multiplier,
node_position_multiplier = node_position_multiplier,
edge_color = c("red")) |>
nodes(
node_size_multiplier = node_size_multiplier,
node_position_multiplier = node_position_multiplier,
self_connection_color = c("red"))Since this is the first ONA network visualization in this chapter, we briefly explain how to read an ONA network.
Node size: In ONA, the node size is proportional to the number of occurrences of that code as a response to other codes in the data, with larger nodes indicating more responses. For example, in this plot, students in the FirstGame condition responded most frequently with discourse about Technical.Constraints.
Self-connections: The color and saturation of the circle within each node is proportional to the number of self-connections for that code: that is, when a code is both what students responded to and what they responded with. Colored circles that are larger and more saturated reflect codes with more frequent self-connections.
Edges: Note that unlike most directed network visualizations, which use arrows or spearheads to indicate direction, ONA uses a “broadcast” model, where the source of a connection (what students responded to) is placed at the apex side of the triangle and the destination of a connection (what students responded with) is placed at its base.
Chevrons on edges: The chevrons point in the direction of the connection. Between any pair of nodes, if there is a bidirectional connection, the chevron only appears on the side with the stronger connection. This helps viewers differentiate heavier edges in cases such as between Technical.Constraints and Data, where the connection strengths from both directions are similar. When the connection strengths are identical between two codes, the chevron will appear on both edges.
plot(set.ona, title = "SecondGame (blue) mean network") |>
units(
points=set.ona$points$Condition$SecondGame,
points_color = "blue",
show_mean = TRUE, show_points = TRUE, with_ci = TRUE) |>
edges(
weights = set.ona$line.weights$Condition$SecondGame,
edge_size_multiplier = edge_size_multiplier,
edge_arrow_saturation_multiplier = edge_arrow_saturation_multiplier,
node_position_multiplier = node_position_multiplier,
edge_color = c("blue")) |>
nodes(
node_size_multiplier = node_size_multiplier,
node_position_multiplier = node_position_multiplier,
self_connection_color = c("blue"))Plot the subtracted network as follows.
# FirstGame and SecondGame subtracted plot
plot(set.ona, title = "Difference: FirstGame (red) vs SecondGame (blue)") |>
units(
points = set.ona$points$Condition$FirstGame,
points_color = "red",
show_mean = TRUE, show_points = TRUE, with_ci = TRUE) |>
units(
points = set.ona$points$Condition$SecondGame,
points_color = "blue",
show_mean = TRUE, show_points = TRUE, with_ci = TRUE) |>
edges(
weights = (colMeans(set.ona$line.weights$Condition$FirstGame) - colMeans(set.ona$line.weights$Condition$SecondGame))*4, # optional multiplier to adjust for readability
edge_size_multiplier = edge_size_multiplier,
edge_arrow_saturation_multiplier = edge_arrow_saturation_multiplier,
node_position_multiplier = node_position_multiplier,
edge_color = c("red","blue")) |>
nodes(
node_size_multiplier = node_size_multiplier,
node_position_multiplier = node_position_multiplier,
self_connection_color = c("red","blue"))To plot an individual student’s network and ONA point, use
set.ona$points.
# first game
plot(set.ona, title = "FirstGame::steven z") |>
units(
points=set.ona$points$ENA_UNIT$`FirstGame::steven z`,
points_color = "red",
show_mean = FALSE, show_points = TRUE, with_ci = FALSE) |>
edges(
weights = set.ona$line.weights$ENA_UNIT$`FirstGame::steven z`,
edge_size_multiplier = edge_size_multiplier,
edge_arrow_saturation_multiplier = edge_arrow_saturation_multiplier,
node_position_multiplier = node_position_multiplier,
edge_color = c("red")) |>
nodes(
node_size_multiplier = node_size_multiplier,
node_position_multiplier = node_position_multiplier,
self_connection_color = c("red"))In this case, both units make relatively strong connections between Design.Reasoning and Data. However, for Unit A (red), the connection is relatively more from Design.Reasoning to Data than the other way around. This indicates that more often this unit responded with Data. In contrast, Unit B (blue) responded more frequently to Data with Design.Reasoning.
A subtracted network can make such differences more salient.
# units difference
mean1 = as.vector(as.matrix(set.ona$line.weights$ENA_UNIT$`FirstGame::steven z`))
mean2 = as.vector(as.matrix(set.ona$line.weights$ENA_UNIT$`SecondGame::samuel o`))
subtracted.mean = mean1 - mean2
plot(set.ona, title = "subtracted network of steven z (red) and Samuel (blue)") |>
units(
points = set.ona$points$ENA_UNIT$`FirstGame::steven z`, points_color = "red",
point_position_multiplier = point_position_multiplier,
show_mean = FALSE, show_points = TRUE, with_ci = FALSE) |>
units(
points = set.ona$points$ENA_UNIT$`SecondGame::samuel o`, points_color = "blue",
point_position_multiplier = point_position_multiplier,
show_mean = FALSE, show_points = TRUE, with_ci = FALSE) |>
edges(
weights = subtracted.mean*2,
edge_size_multiplier = edge_size_multiplier,
edge_arrow_saturation_multiplier = edge_arrow_saturation_multiplier,
node_position_multiplier = node_position_multiplier,
edge_color = c("red", "blue")) |>
nodes(
node_size_multiplier = node_size_multiplier,
node_position_multiplier = node_position_multiplier,
self_connection_color = c("red", "blue")) The connection between Design.Reasoning and Data consists of two triangles, one in blue pointing from Data to Design.Reasoning, the other in red pointing from Design.Reasoning to Data. This indicates that although both units made strong connections between these two codes, the relative directed frequencies are different. Recall that in the ENA subtracted network for the same two units, the connections between Data and Design.Reasoning were basically the same. ONA, by accounting for the order of events, shows that while the undirected relative frequencies were similar, there was a difference in the order in which the two students made the connection.
In addition to visual comparison of networks, ENA points can be analyzed statistically. For example, here we might test whether the patterns of association in one condition are significantly different from those in the other condition.
To demonstrate both parametric and non-parametric approaches to this question, the examples below use a Student’s t test and a Mann-Whitney U test to test for differences between the FirstGame and SecondGame condition.
First, install the lsr package to enable calculation of
effect size (Cohen’s d) for the t test.
install.packages('lsr', repos = "https://cran.rstudio.org")
#> Installing package into '/private/var/folders/5l/hqcrmb1n08b082ycb_ybqlnc0000gp/T/RtmpfNVYez/temp_libpath296341f6e536'
#> (as 'lib' is unspecified)
#>
#> The downloaded binary packages are in
#> /var/folders/5l/hqcrmb1n08b082ycb_ybqlnc0000gp/T//Rtmph5YdLx/downloaded_packages
library(lsr)Then, subset the points to test for differences between the points of the two conditions.
ona_first_points_d1 = as.matrix(set.ona$points$Condition$FirstGame)[,1]
ona_second_points_d1 = as.matrix(set.ona$points$Condition$SecondGame)[,1]
ona_first_points_d2 = as.matrix(set.ona$points$Condition$FirstGame)[,2]
ona_second_points_d2 = as.matrix(set.ona$points$Condition$SecondGame)[,2]Conduct the t test on the first and second dimensions.
# parametric tests
t_test_d1 = t.test(ona_first_points_d1, ona_second_points_d1)
t_test_d1
#>
#> Welch Two Sample t-test
#>
#> data: ona_first_points_d1 and ona_second_points_d1
#> t = -3.7729, df = 41.001, p-value = 0.0005111
#> alternative hypothesis: true difference in means is not equal to 0
#> 95 percent confidence interval:
#> -0.18227713 -0.05517572
#> sample estimates:
#> mean of x mean of y
#> -0.05441628 0.06431015
t_test_d2 = t.test(ona_first_points_d2, ona_second_points_d2)
t_test_d2
#>
#> Welch Two Sample t-test
#>
#> data: ona_first_points_d2 and ona_second_points_d2
#> t = -6.9301e-16, df = 45.45, p-value = 1
#> alternative hypothesis: true difference in means is not equal to 0
#> 95 percent confidence interval:
#> -0.1008208 0.1008208
#> sample estimates:
#> mean of x mean of y
#> -1.727628e-17 1.742362e-17Compute any other statistics that may be of interest. A few examples are given below.
mean(ona_first_points_d1)
#> [1] -0.05441628
mean(ona_second_points_d1)
#> [1] 0.06431015
mean(ona_first_points_d2)
#> [1] -1.727628e-17
mean(ona_second_points_d2)
#> [1] 1.742362e-17
sd(ona_first_points_d1)
#> [1] 0.09754142
sd(ona_second_points_d1)
#> [1] 0.1171941
sd(ona_first_points_d2)
#> [1] 0.1784777
sd(ona_second_points_d2)
#> [1] 0.1679372
length(ona_first_points_d1)
#> [1] 26
length(ona_second_points_d1)
#> [1] 22
length(ona_first_points_d2)
#> [1] 26
length(ona_second_points_d2)
#> [1] 22
cohensD(ona_first_points_d1, ona_second_points_d1)
#> [1] 1.109985
cohensD(ona_first_points_d2, ona_second_points_d2)
#> [1] 1.997173e-16Here, along the x axis (MR1), a two-sample t test assuming unequal variance shows that the FirstGame (mean=-0.05, SD=0.09, N=26) condition is statistically significantly different for alpha=0.05 from the SecondGame condition (mean=0.06, SD=0.12, N=22; t(41.001)= -3.77, p=0.00, Cohen’s d=1.1). Along the y axis (SVD2), a two-sample t test assuming unequal variance shows that the FirstGame condition (mean=-1.73, SD=0.17, N=26) is not statistically significantly different for alpha=0.05 from the SecondGame condition (mean=1,74, SD=0.17, N=22; t(45.45)= 0, p=1.00, Cohen’s d=0.00).
The Mann-Whitney U test is a non-parametric alternative to the independent two-sample t test.
First, install the rcompanion package to calculate the
effect size (r) for a Mann-Whitney U test.
Then, conduct a Mann-Whitney U test on the first and second dimensions.
# non parametric tests
w_test_d1 = wilcox.test(ona_first_points_d1, ona_second_points_d1)
w_test_d2 = wilcox.test(ona_first_points_d2, ona_second_points_d2)
w_test_d1
#>
#> Wilcoxon rank sum exact test
#>
#> data: ona_first_points_d1 and ona_second_points_d1
#> W = 130, p-value = 0.0009533
#> alternative hypothesis: true location shift is not equal to 0
w_test_d2
#>
#> Wilcoxon rank sum exact test
#>
#> data: ona_first_points_d2 and ona_second_points_d2
#> W = 264, p-value = 0.6593
#> alternative hypothesis: true location shift is not equal to 0Compute any other statistics that may be of interest. A few examples are given below.
median(ona_first_points_d1)
#> [1] -0.04307778
median(ona_second_points_d1)
#> [1] 0.09596238
median(ona_first_points_d2)
#> [1] 0.001753116
median(ona_second_points_d2)
#> [1] 0.05862436
length(ona_first_points_d1)
#> [1] 26
length(ona_second_points_d1)
#> [1] 22
length(ona_first_points_d2)
#> [1] 26
length(ona_second_points_d2)
#> [1] 22
abs(wilcoxonR(ona_first_points_d1, ona_second_points_d1))
#> r
#> 0
abs(wilcoxonR(ona_first_points_d2, ona_second_points_d2))
#> r
#> 0.707Here, along the x axis (MR1), a Mann-Whitney U test
shows that the FirstGame condition (Mdn=-0.04, N=26) was statistically
significantly different for alpha=0.05 from the SecondGame condition
(Mdn=0.10, N=22 U=130, p=0.001, r=0.00). Along the y axis
(SVD2), a Mann-Whitney U test shows that the FirstGame
condition (Mdn=0.001, N=26) is not statistically significantly different
for alpha=0.05 from the SecondGame condition (Mdn=0.00, N=22, U=264,
p=0.66, r=0.71). The absolute value of r value in
Mann-Whitney U test varies from 0 to close to 1. The interpretation
values for r commonly in published literature is:
0.10 - < 0.3 (small effect),
0.30 - < 0.5 (moderate effect) and
>= 0.5 (large effect).
Briefly, variance explained (also called explained variation) refers to the proportion of the total variance in a dataset that is accounted for by a statistical model or set of predictors.
In ONA, to represent high-dimensional vectors in a two-dimensional space, ONA uses either singular value decomposition or means rotation combined with SVD. For each of the reduced dimensions, the variance in patterns of association among units explained by that dimension can be computed.
head(set.ona$model$variance,2)
#> MR1 SVD2
#> 0.1367940 0.2736079In our example above, since we used means rotation method, the first dimension is labeled as MR1 and the second dimension is labeled as SVD2.The two dimensions in combination explained about 40% of the variance.
As with any statistical model, greater explained variance does not necessarily indicate a better model, as it may be due to overfitting, but it provides one indicator of model quality.
Briefly, a model’s goodness of fit refers to how well a model fits or represents the data. A model with a high goodness of fit indicates that it accurately represents the data and can make reliable predictions.
In ONA, a good fit means that the positions of the nodes in the space—and thus the network visualizations—are consistent with the mathematical properties of the model. In other words, we can confidently rely on the network visualizations to interpret the ONA model. The process that ONA uses to achieve high goodness of fit is called co-registration, the same as the one used in ENA. The mathematical details of co-registration are beyond the scope of this chapter and can be found in Bowman et al., (2022).
To test a model’s goodness of fit, use
ona::correlations. The closer the value is to 1, the higher
the model’s goodness of fit is. Most ENA models have a goodness of fit
that is well above 0.90.
ona::correlations(set.ona)
#> pearson spearman
#> 1 0.9801173 0.9801799
#> 2 0.9801431 0.9759160Another approach to evaluate an ONA model is to confirm the alignment between quantitative model (in our case, our ONA model) and the original qualitative data. In other words, we can return to the original data to confirm that quantitative findings give a fair representation of the data. This approach is an example of what’s called as closing the interpretative loop in Quantitative Ethnography field (Shaffer, 2017).
For example, based on our visual analysis of the network of “SecondGame::samuel o” in previous section, we are interested in what the lines are in the original data that contributed to the connection from Performance.Parameters to Design.Reasoning.
Let’s first review what “SecondGame::samuel o” ONA network looks like. Based on the connection direction and strength from Technical.Constraints to Performance.Parameters, we would expect to see more examples of Samuel responded with “Design.Reasoning” to “Performance.Parameters”, than the other way around.
plot(set.ona, title = "SecondGame::samuel o") |>
units(
points=set.ona$points$ENA_UNIT$`SecondGame::samuel o`,
points_color = "blue",
show_mean = FALSE, show_points = TRUE, with_ci = FALSE) |>
edges(
weights = set.ona$line.weights$ENA_UNIT$`SecondGame::samuel o`,
edge_size_multiplier = edge_size_multiplier,
edge_arrow_saturation_multiplier = edge_arrow_saturation_multiplier,
node_position_multiplier = node_position_multiplier,
edge_color = c("blue")) |>
nodes(
node_size_multiplier = node_size_multiplier,
node_position_multiplier = node_position_multiplier,
self_connection_color = c("blue"))To do so, we use view() function and specify required
parameters as below.
This is going to activate a window shows up in your
Viewer panel. If it is too small to read, you can click on
the “Show in new window” button to view it in your browser for better
readability.
view(accum.ona, # the object stored our connection accmulation reults in 4.3.6
wh = c("SecondGame::samuel o"), # the unit we are intersted in
units.by = c("Condition", "UserName"), # consistent with in 4.3.1
conversation.by = c("Condition", "GroupName", "ActivityNumber"), # consistent with in 4.3.3
codes = c("Performance.Parameters", "Design.Reasoning"), # codes of choice
window = 7) # consistent with in 4.3.4In the Viewer panel, hover over your cursor on any of
the lines that are in bold, a size of 7 lines rectangle shows up,
representing that in a moving stanza window of size 7, the referent line
(the line in bold) and its preceding 6 lines. The 1 and 0 in
Technical.Constraints column and Design.Reasoning column shows where the
connections happened.
Notice that here we are viewing the same qualitative example as in section 3.7.3 in ENA. In line 2477 Samuel shared his [Design.Reasoning] about “mindful of (the) how one device scores relative to other ones”, as a response to what Casey said in line 2476 about [Performance.Parameters] “not one source/censor can be the best in every area so we had to sacrifice certain attributes”, as well as what Jackson said in line 2475 about safety as one of the [Performance.Parameters] “when it came to the different attributes, i think that all were important in their own way but i think safety is one of the most important”.
Here, ONA was able to not only capture the occurrence between code Design.Reasoning and Performance.Parameters as ENA did, but also represent the connection direction from Design.Reasoning to Performance.Parameters
To project the ONA points from one model into a space constructed
with different data, replace the rotation.set parameter of
model. In the example below, an “expert” model is developed
using the SecondGame units and the FirstGame (novice) units are
projected into that space.
data = ona::RS.data
#expert data
exp.data = subset(data, Condition == "SecondGame")
#novice data
nov.data = subset(data, Condition == "FirstGame")
#shared unit cols
units = c("UserName","Condition","GroupName")
#shared code cols
codes = c(
'Data',
'Technical.Constraints',
'Performance.Parameters',
'Client.and.Consultant.Requests',
'Design.Reasoning',
'Collaboration')
#shared hoo
hoo = conversation_rules(
(Condition %in% UNIT$Condition & GroupName %in% UNIT$GroupName))
#expert accum
accum.exp = contexts(exp.data, units_by = units, hoo_rules = hoo) |>
accumulate_contexts(codes = codes,
decay.function = decay(simple_window, window_size = 7),
return.ena.set = FALSE, norm.by = NULL)
#expert model
set.exp = model(accum.exp)
#novice accum
accum.nov = contexts(nov.data, units_by = units, hoo_rules = hoo) |>
accumulate_contexts(codes = codes,
decay.function = decay(simple_window, window_size = 7),
return.ena.set = FALSE, norm.by = NULL)
#novice model
set.nov = model(accum.nov)
# projecting novice data into expert space
set = model(accum.nov, rotation.set = set.exp$rotation)
plot(set, title = "novice data into expert space") |>
units(
points = set$points,
show_mean = TRUE, show_points = TRUE, with_ci = TRUE) |>
edges(
weights = set$line.weights) |>
nodes(
self_connection_color = "red",
node_size_multiplier = 0.6)