Protein engineering is a burgeoning field within the life sciences promising targeted therapeutics, enhanced agricultural yield, and more efficient manufacturing.

Executive Summary

 

Introduction

Protein engineering is a burgeoning field within the life sciences promising targeted therapeutics, enhanced agricultural yield, and more efficient manufacturing. Various models and analysis paradigms employed by scientists and engineers leverage statistics and cutting-edge machine learning models to guide desirable functional changes. While notable advancements have been made concerning modeling protein tertiary structure as AlphaFold’s attention network has accomplished, there is room for simpler graphical models with better feature extractability to quickly inform scientists of key functional associations.(Senior, 2020).

Biological Background

Proteins are a polymer consisting of amino acids (of which there are 20) in a linear chain. An amino acid is composed of one nitrogen and two carbon atoms and is bound to various hydrogen and oxygen molecules, as shown in Figure 1. The central carbon Cα is linked to the unit “R” or residue, which distinguishes the amino acid. Amino acids bind through the loss of water molecules and the remaining parts of the amino acids are known as amino acid residues. Amino acids bind to form chains of hundreds to thousands of amino acids, forming the primary structure of proteins.

Figure 1: Amino acid structure. Retrieved from https://study.com/academy/lesson/what-is-amino-acid-residue.html

Amino acids in the chain can also interact with other non-adjacent amino acids in the same chain. This can cause the folding of the amino acid chain and lead to varying three-dimensional structures (secondary and tertiary structures). The two common forms of secondary structure include alpha helices and beta sheets. Proteins are essential in every cellular process. Many proteins are functional as monomers. Other proteins often form complexes (protein-protein interaction) to achieve specific functions. This is known as the quaternary structure of proteins. The four levels of protein structures are visually represented in Figure 2.

Figure 2: Protein structure: Primary, secondary, tertiary, and Quaternary. Retrieved from https://www.thoughtco.com/protein-structure-373563

Protein-protein or residue-residue interactions are the heart of biological processes. They give the protein its structure, which brings us to the key idea of biology: “structure equals function”. Thus, it is crucial to be able to identify these interaction sites or interface residues, as they can indicate the functionality of proteins. In this case, a protein can be modeled graphically where nodes are referred to as the 3D residue position, and the edges in the graph illustrate the spatial neighborhood of the residue.

Modeling

There are many tasks involving predicting the large numbers of variables that depend on pairwise associations. The method of a structured prediction is critical in graphical modeling and a combination of classification. (Athar, 2018). These pairwise associations can classify compact multivariate data, thus performing predictions that use a large set of individual features. Conditional random fields (CRFs), a popular probabilistic method in structured prediction, are one flavor of these graphical models. It is worth noting that CRFs have very wide applications as they are used in computer vision, natural language processing, and bioinformatics. The available methods for inference in estimating CRFs entail the practical issues required in large-scale CRFs implementation. Briefly, we can define CRFs as a statistical model method applied in pattern recognition and machine learning for structured prediction.

CRFs are a component of the standard mathematical modeling method used in certain navigational softwares where it enjoys popularity identifying the direction and orientation of the device. The models additionally assist in calculating miles traveled while offline (Xu, 2015). Other computer vision applications have demonstrated that Neural Nets with CRF layers have predictive capabilities rivaling heavier Graphical Neural Networks with less computational resources on a notoriously difficult dataset called tanks and temples(Bao, 2019). CRFs are a type of non-targeted graphical model. Usually, this enters the code in the captured relationship between the visuals and creates a consistent interpretation. It is often used to label or subdivide consecutive facts, text, or Biological sequences. In particular, CRFs model key business relations, genetic discovery, and peptide information to inform organizations. In computer vision, CRFs are often used for object recognition and image classification. There are several types of conditional Random fields. One is higher-order and semi- Markov, but there are also latent –dynamic conditional random fields (Suraksha, N. M, 2017).

CRF Types

Higher Order and Semi-Markov

CRFs can be extended to higher-order models using individualization depending on the consistency variety of previous variables. Learning and inclination work with great success in small amounts given that their calculation costs will increase significantly. The mainline models of the established forecast, consisting of a dependent person assisting with the Vector program, can be seen as a training opportunity for CRFs. An alternative version of the CRF is a random area with semi-Markov conditions (semi-CRF), which compares the duration components. This provides significant learning capabilities at a fraction of the compute time for GNNs. Comment by Alexander Larsen: Expand upon this. More options/ types

Dynamic

Dynamic brief-term dynamic fields or brief-time period dynamic fields are the CRF technique of consecutive marking bonds. They may be a dynamic hidden model that can be effective in discrimination. In Latent-Dynamic Conditional Random Field(LDCRF), as with all series tagging tasks, you’re given a series of thoughts x = x₁… xₙ, the main trouble the model needs to clear the way to share the series of labels y = y₁… yₙ in a single complete set of Y. labels in place of without delay modeling P (y | x) like an ordinary line chain. CRF could make, hard and rapid with hidden flexibility “inserted” among x and y the possible use of chain regulation: This allows for the taking pictures of a hidden structure between visuals and labels. While LDCRFs can use quasi-Newton strategies, a special version of the perceptron set of rules referred to as modified perceptrons is also designed for them, based on the built-in perceptron set of rules. These types get packages in pc viewing, especially contact recognition from video streaming and in-depth analysis. Comment by Biraj Shrestha: wha does the boxes after x mean, is it a symbol for something, please specify.

 

 Comment by Biraj Shrestha: added this to illustrate the nodes and edges

Fig 3: Simple representation of a network, nodes representing components and edges representing interactions. https://www.sciencedirect.com/science/article/pii/S2001037014000233

Our approach is based on conditional random fields (CRFs) proposed by Lafferty which are related to the probabilistic methods. A CRF can use a connected graph as opposed to other statistical models, such as hidden Markov models, which only have edges between adjacent nodes. This makes CRFs a better predictor of functionality as many residues in proteins interact with other residues besides their immediate neighbors. Linear-chain CRFs, like Hidden Markov Models, only impose dependencies on the previous element and it can not represent the three-dimensional structure of a protein. Our project is concentrating on graphical CRFs where we can impose dependencies on arbitrary elements. The project goal is to take a family of proteins and create a graph CRF that can act as a scoring system for new sequences that assesses whether the new sequences have the same functionality as the family. https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.0030119

Pros and Cons

Advantages

The conditional random fields offer many advantages over Markov’s hidden models and the stochastic grammar system for such functions, including illuminating the strong independent. Assumptions were made on those models. In addition to the 22 basic catches of the many entropy Markov fashions and the different models of Markov’s discrimination, the unconditional random fields are based primarily on targeted fashions, which may favour countries with a few consecutive provinces. It can measure the parameters of conditional forums and evaluate the effectiveness of the following models in the Hidden Markov Model(HMM) and Maximum Entropy Markov Model(MEMMs) in the practicalities and language of herbs.

This concept explores other techniques for measuring field parameters for random, newly introduced versions that can label and separate sequential data. The intellectual and practical risks of learning strategies used in current CRF textbooks are touched on. We assume that standard pricing strategies lead to more advanced performance than CRF training algorithms. Experiments use a set of popular content determination records that verify this to be true. This is a surprisingly promising result, showing that such parameter measurement techniques make CRFs an effective and green desire to write sequential data, as well as a framework of sound and objective beliefs.

Conditional Random Fields (CRFs) is an unconventional graphic style, a completely different case associated with state-of-the-art information technology. The biggest advantage of CRFs is their amazing flexibility that includes a wide range of competitive, impartial entry functions. Facing this freedom, however, the important question remains: what skills should be used? This approach is based entirely on the command combination integration factor that can greatly increase the risk of entry conditions presented in the model. Permissions for automated inputs do not work with precise precision, and high precision parameters depend. Still, the use of large groups and greater freedom of flexibility of atomic input may be associated with the challenge. This approach applies to online CRFs and other CRF structures, including Relational Markov Networks. It is linked to the acquisition of clique templates and can be understood by a supervised form of knowledge. It provides the results of the test on the issuance of a fictional business and the obligations of word separation.

Limitations

The most obvious disadvantage of CRF is the high computational complexity of algorithm training. This makes it very difficult to retrain the model as new school learning data samples become available. In addition, CRF now does not make drawings with unknown expressions, meaning phrases that were no longer in the sample of educational facts.

Circles and rectangles correspond to labels (Y) and comments (X). It is very important to remember the hyperlink regime’s power changes within a simultaneous version for visibility with smart home layouts. Such features are difficult to stand on in HMM because they create opportunities but can be addressed with the help of CRFs. CRF can be represented as an indirect graph G = (V, E). The distribution of non-target graph opportunities is calculated with the help of the addition of maximal groups of three c ε cliques V of the graph. Graphic fashions in natural language processing. Even though those examples are popular, they work both to make the file explanations in the previous section and to show other ideas that will also arise in our discussion of conditional random fields. The unique interest in Markov (HMM) is a hidden form because it is several miles in line with the CRF chain. Graphic fashions in natural language processing. Even though those examples are popular, they work both to make the file explanations in the previous section and to show other ideas that will also arise in the discussion of conditional random fields. The unique interest in Markov (HMM) is a hidden form because it is several miles in line with the CRF chain. Comment by Biraj Shrestha: What does circles and rectangles refers to? is it referring to a figure? if so please add a figure. Comment by Biraj Shrestha: And again, in-text citation should be on each of the paragraph. Comment by Biraj Shrestha: can you please add a graph thats its referring to.

PyStruct’s desire to provide a definitive cause for the implementation of preferred preferences and predictive methods, each designed for physicians and as a basis for researchers. Written in Python also synchronizes paradigms and types from seamless Python medical network integration with other activities. Key phrases: systematic predictions dom fields, Python. PyStruct aims to be a properly prepared and predictable studying library. (Cao (2020). It presently uses the most effective max-margin and perceptron strategies, but other algorithms may additionally observe. The getting-to-know algorithms used in PyStruct have exclusive names, frequently used freely or one by one in one-of-a-kind communities. Common names are conditional random fields (CRFs), high-degree random fields (M3N), or vector help equipment.

There are several things that we train them before feeding the actual data; this includes:

Rating: The facts can also contain adjectives with combos on various scales such as greenbacks, pounds, and income. Many ways to manage devices are a true symbol of having the same scale that ranges from 0 and 1 at the lowest and largest price than the feature provided. Remember any measurement you may need to achieve.

Decay: There may be factors that create a complex concept that can be very helpful in reading the gadget while cutting it into key parts. For example, a day that may have additional day and time additions that can be further cut. It probably works best for an hour a day to solve a problem. Remember what factor decay you can do.

Integration: There may be skills that can be directly integrated into one aspect that may have more purpose in the problem you are trying to solve. For example, there may be instances of information every time a consumer logs into a device that is not included in the calculation of the login number that allows for additional time to be lost. Keep in mind what type of feature integration you may want to achieve.

Statistical Models

In this project, we are focusing on conditional random fields which are a class of statistical modeling methods. Statistical models use mathematical models and statistical assumptions to generate sample data and make predictions for populations. In simple language, it can be considered as a pair (X,P) where X represents the set of observations and P is the set of possible probability distributions on X. The process of evaluating the parameter in the statistical model is known as training. In order to estimate how the model is expected to perform, we distinguish the data into two sets: training data and testing data. Training data set is used to create the model and the test or validation data set is used to test the performance of the final model.

Graphical Models

Graphical models are a class of statistical models which is represented via a graph and mathematically denoted by a pair G = (V, E). Where V is nodes and E is edges. There are two types of graphical models: directed graphical models and undirected graphical models. In directed graphical models, the edges of the graph have directions (Bayesian network), whereas in undirected graphical models, the edges carry no directional information (Markov networks). A clique C of an undirected graph is the maximal complete subgraph. The figure (xx) shows an undirected graph with three maximal cliques, {1, 2, 3, 4}, {4, 5} and {5, 6}.

Figure ##: Example of an undirected graph with three maximal cliques.

Directed graphical models describe how label vectors can generate feature vectors probabilistically. For this reason, they are known as generative models. Contrastingly, undirected graphical models describe how to assign feature vectors to label vectors. They are also known as discriminative models. The figure below describes the analogy between different graphical models such as naive Bayes, logistic regression, HMMs, linear-chain CRFs, generative directed models, and general CRFs. The main difference between naive Bayes and logistic regression is that naive Bayes is generative model, meaning that it depends on the joint distribution p(y, x), whereas logistic regression is discriminative model, meaning that it depends on the conditional distribution p(y|x). The relationship between logistic regression and generative models mirrors the relationship between Hidden Markov Models (HMMs) and linear-chain conditional random fields.

Figure ##: The relationship between naive Bayes, logistic regression, HMMs, linear-chain CRFs, generative directed models, and general CRFs. Retrieved from https://people.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf

Hidden Markov Models

Hidden Markov Model (HMM) is a stochastic model based on sequential data. It contains a Markov chain with a finite number of hidden events (emission states) and observed events. In HMM, each hidden state Yi (except Y1) depends only on the previous state Yi-1, i = 2, 3, 4…., n and each observed state Xi depends only on the current state Yi, i = 1, 2, 3, 4…., n.

Figure x: Hidden Markov Model with hidden states Yi and observed states Xi. Retrieved from https://www.alibabacloud.com/blog/hmm%2C-memm%2C-and-crf%3A-a-comparative-analysis-of-statistical-modeling-methods_592049

There are three parameters in HMM; starting probabilities P(y1), transition probabilities P(yi|yi-1), i = 2, 3, 4…., n, and emission probabilities P(xi|yi), i = 1, 2, 3, 4…., n. The probability of an observed state x is labeled by a hidden state y is given by:

 

with P(y1|y0) = P(y1)

The limitation of this model is that the observed state xi only depends on the emission state yi. When the model is predicting the value for yi, it cannot directly consider knowledge from the observed variables xi. http://cs.tulane.edu/~aculotta/pubs/culotta05gene.pdf

Conditional Random Fields

A conditional random field (CRF) is an undirected graphical model. It can be considered as a generalization of the hidden Markov model, meaning we can consider the conditional distribution p(y|x) that results from the joint distribution p(y, x). The difference between HMM and CRF is that CRF calculates conditional distribution and HMM calculates joint distribution. According to Lafferty, “Let x be an observation over data and y = (y1,y2,…, yn) one of the possible label sequences. Moreover, F = {fk, k = 1, 2, . . . , K} denotes a set of real-valued feature functions with a weight vector Λ = {λk}k=1K . Then a linear-chain conditional random field takes the form

 

where the normalization factor

 

.”

Here, the normalization factor Z(x) sums over all possible state sequences, which is an exponentially large number of terms. Real world observations generally have multiple interacting features and dependencies, making it difficult to model the distribution of P(x). Use of the independent assumption in HMMs is not warranted, and thus discriminative models like linear chain CRFs are preferred. Linear chain CRFs, however are seen as only a linear structure, which is not sufficient for this project. Latent node graphical CRF models were developed and the graphical relationships investigated.

The graphical CRF can be defined as

 

Where each factor is parameterized as:

 

And the normalization function is

 

In graphical CRF, let us consider G to be the factor graph over Y. Then a conditional random field p(y|x) for any fixed x factorizes according to G. We partition the factors of G into C = {C1, C2, …, Cp), where each Cp is a clique template. Each clique template is a set of factors that has a equivalent set of adequate statistics {fpk(xp, yp) and parameters . https://people.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf

Protein multiple sequence alignment (Sanju)–in progress

Protein multiple sequence alignments are an essential tool for protein structure and function prediction. Distantly related sequences of proteins can be identified and aligned using multiple sequence alignment. It can also be used to identify known sequence domains in new sequences. Multiple sequence alignment uses a position-specific scoring matrix (PSSM), allowing for the degree of conservation at various positions to be determined.

Multiple sequence alignments work by analyzing if residues in a given column are homologous or play a common functional role. A single residue mutation in a column of an MSA can influence a compensating mutation in a different column, indicating that the two residue sites are coevolved. The mutated residue sites are thus key for determining protein-protein interactions. Thus, the first step to determining this is to determine the co-evolved sites in an MSA. Neighboring residues in an amino acid sequence are connected by peptide bonds. These form the primary structure of proteins. Residues that are not neighboring may also connect through hydrogen bonding or di-sulfite bonds. These bonds are what allow for the protein to form three-dimensional structures. It is the structure formed here which is critical to the stability and functionality of the protein.

To model these interactions, the latent node graph CRF model is chosen. In the 3D structure of a protein, nodes represent the residues in the protein and edges represent the spatial neighborhood among the residues. The latent node graphical CRF model involves interactions with variables that are not observed during training. Hidden causes of the data are often modeled, making it easier to learn about the actual observations.

· Detailed explanation of Pystruct (Sanju)

Software

Pystruct

In this project, we are using Pystruct software as it fits the desired capabilities stated in the above CRF section. Pystruct is a Python library, which is based on general Conditional Random Field models (CRF). Python provides a general implementation of standard structured prediction methods, which is defined as maximizing the compatibility function between inputs (x) and possible labels (y) to make a prediction, f(x), as shown in the following equation. Comment by Alexander Larsen: Add caveat that pystruct fits the desired capabilities stated above in the CRF explanation. It’s a good software to accomplish this. Comment by Sanju Wagle: Added

 

 Comment by Sanju Wagle: will write the formula.

Where; y is a structured label, Ψ is a joint feature function of x and y, and θ are parameters of the model. The parameters support algorithms for structural support vector machines (SSVMs), subgradient methods for SSVMs, block-coordinate Frank-Wolf (BCFW), the structured perceptron, and latent variable SSVMs.

The joint feature function and encoding of the problem structure is computed by model classes. The structure of the joint feature function determines the hardness of the maximization. Pystruct is capable of implementing a wide range of models including CRFs. External libraries, such as OpenGM and LibDAI are used to maximize the possible labels. Use of external libraries allows for a wide range of optimization algorithms including QPBO, MPBP, TRWs and LP. https://jmlr.csail.mit.edu/papers/volume15/mueller14a/mueller14a.pdf

The training models: Sanju—in progress Comment by Alexander Larsen: Briefly talked about in the discussion which might be a better section for it. Please add your words about the frank wolfe work down there!

· SSVMs

· Frank-Wolfe

· OneSlack

· SubgradientSSVM

 

Method

Protocol

Pystruct can be installed directly using pip if using an older version of python <3.2; otherwise the library should be taken from the pystruct/pystruct github page. After the library has been installed using the included setup.py file, the library can be imported directly in a python script or jupyter notebooks. If this version does not work due to having windows or macOS, we created an updated version that is stored under the github user maxpwilson . After installation of pystruct, users can then download our project from here.

Figure X. A screen shot of the frontpage of our github repository.

The notebooks directory contains two notebooks and their supporting files. The first file, control_pull.ipynb can be used in conjunction with a Genbank file to find, download, and align control sequences. The built-in method employs mafft to speed up computation and is versatile for further development with the program’s addfragments and keeplength options allowing for new sequences to be added to the existing structure(Katoh, 2013).

Fig X. A test printout of the control retrieval script showing the flags to input your email and ncbi_api_key as well as limit the search space in the PullQCGenes class from the CRFSeqs program. The result of this script can output to a csv for later usage in pystruct.

A demo for using LatentNodeCRFs in pystruct and preprocessing the data is included at LatentNodeCRF_demo from within the notebooks file. The start of the notebook delineates a section describing how to format an MSA, generate a one-hot encoding for each amino acid within a sequence and then generate a list of latent edges for every pairwise interaction.

After formatting the data, pystructs model LatentNodeCRF can be instantiated within a learner SSVM such as NSlackSSVM which can then be used as a base ssvm for the LatentSSVM learner to fit the properly formatted data to.

 

 

 

A B 

Fig X. (A) showing the first steps in formatting the msa in character matrix array and then (B) turning that input into a one hot encoded array with all associated edges and latent features.

Data Format

Pystruct’s data format rigidity required a specific format for data entry. Unlike the base GraphCRF, the LatentNodeCRF and EdgeFeatureGraphCRF pystruct models require an array of three arrays and matrices. The first matrix is a one-hot encoded matrix of the gapped amino acid residue as the core data for the model. The second position required a combinatorial space of all pairwise relationships possible. An odd caveat is that the LatentNodeCRF and EdgeFeatureGraphCRF required these combinations to be the transposed array of one another. The third array can be a singular integer as a constant for a 1-D array Y setting for the LatentNodeCRF or a 2×2 matrix for the EdgeFeatureCRF often used to denote the extra weight of neighboring pixels in an image. The Y format-dependent values can be in a singular point for Latent models and must be in an amino acid length array for the GraphCRF and EdgeFeatureGraphCRF models.

	X	Y
	Pos 1 (nodes)	Pos2 (edges)	Pos 3 (misc)
GraphCRF	N x 22	2 x	–	Nodes length array per replicate
EdgeFeatureGraphCRF	N x 22	2 x	2 x 2	Nodes length array per replicate
LatentNodeCRF	N x 22	x 2	1 x 1*	1 label per replicate

Tabel X . shows the input requirements for pystruct where Position 1 is a one-hot encoded array of amino acids where N is the length of the AA sequence and 22 is the possible AAs that could be within that slot. Position 2 is a list of all possible pairwise edges which are unusually required to be transposed for the latent model. The third position, if applicable, is a metric of adding weights and additional features to the edges. In the latent model, the third position corresponds to the latent states possible and need not necessarily be a 1×1 array per replicate.

SSVM Parameters

The set parameters for all models were fairly consistent. A maximum of 200 training iterations were allowed and preliminary tests showed that lowering this number eventually lowered the models predictive accuracy. The regularization parameter (C) was set to 100 for a strict penalty to be assigned to avoid overfitting based on recommendation from the pystruct source code.

Results

Control Quality

Adk-lid, short for Adenylate kinase, is a conserved protein domain across many species including Streptococcus which is a mare deeply characterized genus rife with multiple gene features for an outgroup comparison. We were able to curate an assortment of 18 gene features totalling 373 sequences that were greater in length compared to the adk-lid domain sequences. After receiving the fasta sequences an alignment, performed with mafft v7.487 (2021/Jul/25) using the default parameters of each cluster was performed and a Position Specific Scoring Matrix was calculated and graphed to assess the quality of the data pulled. A position specific scoring matrix shows the sequence position in the x-axis and the range of possible amino acids in the y-axis. A handful of chosen alignments showed a mix of conservation and diversity amongst the sequences.

The quality of the MSA can be visually observed by observing a heatmap of the position-specific scoring matrix fig x.x

  

Fig: Snapshot of the portion of the control sequence genes showing the number of sequences in the file and the conservation per AA residue elucidated by the position-specific scoring matrix.

Model Training

Using pystruct and the protocol shown above, we were able to use the Latent Structured Support Vector to train a Latent node graphical CRF to high degrees of accuracy. NSlackSSVMs, OneSlackSSVMs, and FrankWolfeSSVMs were evaluated as the base SSVM for the latent learner of which the Slack methods trained to 100% and the FrankWolfe method which had a predictive power of 91.1% successfully predicted scores. The fastest SSVM model was the NSlackSSVM which was 3.9X faster than the second performer, OneSlack and 160X faster than the FrankWolfe learner. Due to the similar results between the top two performers, the faster of the two models was chosen for further examination.

Fig X. Results from the training show that the FrankWolfeSSVM was significantly slower than the other models and has lower performance.

Attempts to fish out the actual pairwise associations (Max)

 

· All the graphs generated

· The model

· CSV

 

Discussion

In SSVM, the joint feature function Ψ represents the relation between x and y. Latent variable SSVMs are generalizations of SSVMs, where joint feature function Ψ(x, y) with an extra argument h to Ψ(x, y, h) to describe the relation between input x, output y, and latent variable h. Comment by Sanju Wagle: We can add this info somewhere in the discussion.

Conditional random fields is a discriminative model i.e it models the conditional probability P(Y/X) which is best suited to predict the tasks where the current position is affected by the contextual information or state of the neighbors. Unlike HMM and MEMM which are a directed graph i.e. it directly models the transition probability and calculates the probability of co-occurrence, CFR is an undirected graph and it calculates the normalization probability in the global scope. concentrating on graphical CRFs where we can impose dependencies on arbitrary elements. In this project we developed a graphical CRF based on the latent node CRF that would score the chances of the new sequences having the same functionality as its family. Comment by Biraj Shrestha: will add citation later

Learner

Convex Optimizers

After performing the trainer comparisons it was clear that the NSlack learner was faster and just as precise if not more precise than the other models. NSlack and OneSlack learners were equivalent in performance, likely due to the underlying design used by both. The slack methods both employ the crxopt which is a python package whose name is a portmanteau of convex and optimization(Andersen, 2011). Cvxopt is likely the reason their performance is far superior as the Frank-Wolfe algorithm-based learner is a similar type of convex optimizer commonly referred to as the conditional gradient method (Kolter, 2019). The difference lies in the fact that the Frank-Wolfe implementation was made by the pystruct designer and does not have the C-based speed or “smart” criteria constraint check which will prematurely check if an optimization step lowered the predictive capabilities in the code. The general speed of the NSlack method is a strongly desirable trait as the protein number and length become increasingly large.

Control Alignments

One key limitation of the study design was that we did not have access to versions of adk-lid that were non-functional. A learner trained on such a model would have had a strong differentiating power if non-functional adk-lid protein controls were unavailable. In an attempt to learn some of the general common pairwise associations, we made an optimistic assumption that the learner would have had just enough noise within the control and target groups to learn meaningful pairwise associations within the target group. We opted for an alignment within singular genes to reduce the size of the alignment and avoid hyper-gappy arrays which likely would have had large gapps between gene clusters resulting in gap locations being the primary learned differentiating feature. Training a model on an alignment of control and target groups theoretically could have been a fruitful endeavor but we would have lost the structure of the initial alignment. In future experiments we would try this total alignment with severe penalties for gap extension, forcing the Needleman wunsch implementation present in mafft to create the most compact alignment possible for training.

 

· Also discuss the possibilities if we had retrieved the pairwise

· Like easy to train with small data set

· Interpret each of the results

· Problems that we faced

· Future plans/recommendations

· Conclusion

· Illustrating the purpose of the CRF

· Summarizing the results and finding

· Future recommendations

 

References: (ordered)

· Agrawal, A., Amos, B., Barratt, S., Boyd, S., Diamond, S., & Zico Kolter, J. (2019). Differentiable convex optimization layers. Advances in Neural Information Processing Systems, 32 (NeurIPS).

· Katoh, K., & Standley, D. M. (2013). MAFFT multiple sequence alignment software version 7: Improvements in performance and usability. Molecular Biology and Evolution, 30(4), 772–780. https://doi.org/10.1093/molbev/mst010

· Luo, X., Li, H., Yu, Y., Zhou, C., & Cao (2020). Combining in-depth features and activity context to improve recognition of activities of workers in groups. Computer‐Aided Civil and Infrastructure Engineering, 35(9), 965-978.

· Meunier, J. L. (2017, November). PyStruct extension for typed crf graphs. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) (Vol. 4, pp. 5-10). IEEE.

· Senior, A. W., Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., Žídek, A., Nelson, A. W. R., Bridgland, A., Penedones, H., Petersen, S., Simonyan, K., Crossan, S., Kohli, P., Jones, D. T., Silver, D., Kavukcuoglu, K., & Hassabis, D. (2020). Improved protein structure prediction using potentials from deep learning. Nature, 577(7792), 706–710. https://doi.org/10.1038/s41586-019-1923-7

· Suraksha, N. M., Reshma, K., & Kumar, K. S. (2017, June). Part-of-speech tagging and parsing of Kannada text using Conditional Random Fields (CRFs). In 2017 International Conference on Intelligent Computing and Control (I2C2) (pp. 1-5). IEEE.

· Xu, M., Du, Y., Wu, J., & Zhou, Y. (2015). Map Matching Based on Conditional Random Fields and Route Preference Mining for Uncertain Trajectories. Mathematical Problems in Engineering, 2015. https://doi.org/10.1155/2015/717095

· Xue, Y., Chen, J., Wen, W., Huang, Y., Yu, C., Li, T., & Bao (2019). Mvscrf: Learning multi-view stereo with conditional random fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 4312-4321).

· Yu, B., & Fan, Z. (2020). A comprehensive review of conditional random fields: variants, hybrids and applications. Artificial Intelligence Review, 53(6), 4289-4333.

· Zia, H. B., Raza, A. A., & Athar (2018). Urdu word segmentation using conditional random fields (CRFs). arXiv preprint arXiv:1806.05432.

· Zhong, Z., Li, J., Clausi, D. A., & Wong, A. (2019). Generative adversarial networks and conditional random fields for hyperspectral image classification. IEEE transactions on cybernetics, 50(7), 3318-3329.

 

 

 

 

 

1