Automatic Timeline Generation - Stanford University

Automatic Timeline Generation Jessica Jenkins Josh Taylor CS 276b Corpus Subset of TDT-3 text news articles

875 articles 49 topics (collections of related articles) 6-30 articles per topic Sources: ABC, APW, CNN, NBC, NYT, VOA Focus on sentence-level event detection within a topic Labeling / Data Processing Manual annotation for evaluation

10 topics, 120 articles Decide on a set of events for each topic Annotate each sentence in each article with a set of relevant topic events Sentence boundary detection MXTerminator: maximum entropy classifier (Reynar and Ratnaparkhi) Data Processing

Sentences tokenized with case-folding, punctuation stripping, Porter stemming, and English stop-word removal Part of Speech tagging Stanford Log-linear PoS Tagger (Toutanova and Manning) Noun Phrase Chunking BaseNP Chunking (Ramshaw and Marcus) Only retained verbs for non-NP terms

Evaluation Precision and recall analogs Defined relative to set of annotated events NU-precision: count of sentences labeled with novel events (relative to higherranked sentences) over number of selected sentences NU-recall: number of events in selected

sentences over number of events in topic Event Selection Methods Language model-based sentence scoring (Allan, Gupta, Khandelwal) Useful score: measure of how likely a sentence is to be on-topic. Novel score: indication that sentence content is unlike previous sentences

Sentences ranked using a weighted blend of their Useful and Novel score Sample Timeline Top 5 sentences by score, ordered chronologically:

As Mr. Sharif left the White House, he said the talks focused on the India-Pakistan dispute over the Kashmir region and U.S. concerns about nuclear arms tests conducted by India and Pakistan. In 1989, Pakistan paid the U.S. close to $700 million for 28 F-16 fighter planes. But Pakistan was never refunded for the F-16s. In return, Pakistan will withdraw its claim to the F-16 aircraft. Delivery of the jets was stopped in 1990 because of the U.S. arms embargo against Pakistan because of its nuclear program. Event Selection Methods Event discovery through clustering

Similar to text summarization Each cluster represents an event Hierarchical agglomerative clustering with sparse vectors K-means (k=5,10,20) with and without dimensionality reduction Clustering Results

Tried using group average HAC without dimensionality reduction Representative sentence chosen for maximal cosine similarity with cluster centroid Underwhelming results with respect to evaluation metrics Sample Timeline 2 Top 5 useful cluster representatives, taken from 15

clusters, ordered chronologically: Mr. Clinton also said he and Mr. Sharif would try to resolve a dispute over a canceled sale of U.S. fighter aircraft. Mr. Clinton wants India and Pakistan, which each conducted nuclear tests last May, to sign the Comprehensive Test Ban Treaty as they have indicated they would do. All of you know my concern to do everything we can to end

the nuclear competition in South Asia which I believe is a threat to Pakistan and India and to the stability of the world. The delivery was blocked under a 1990 law barring direct U.S. military sales to Pakistan because of its development of nuclear weapons. He met with Pakistan's Prime Minister Omar Sharif. Clustering in Progress Using SVD dimensionality reduction Noun phrase features with part-ofspeech filtering

Representative sentences chosen from each cluster based on Useful score Problem Areas Small annotated corpus Few annotated events per topic give a coarse evaluation Weak correspondence between clusters and labeled events Unexplored Possibilities

Detection and exploitation of temporal features in sentences Clustering with different sentence similarity measures WordNet-derived semantic distance

Recently Viewed Presentations

  • Arc Lengths, Sectors, and Rotational Speeds

    Arc Lengths, Sectors, and Rotational Speeds

    Arc Lengths, Sectors, and Rotational Speeds. Dr. Shildneck. Fall, 2015. Arc Length. Recall from the definition, the formula for a radian, is the arc length divided by the radius. This formula will allow us to solve problems that involve the...
  • Exchange Online Protection: Notes from the field

    Exchange Online Protection: Notes from the field

    Exchange Online Protection implementation and configuration assistance. 1 - 5 days of engagement over a period of 90 days. Administrator training on Exchange Online Protection. Advise customer on service best practices. Eligibility. Net new customers who purchase 1000+ seats EOP...
  • Basic Sampling Issues

    Basic Sampling Issues

    Developing a Sampling Plan. Step 1: Define the Population of Interest. The 1st issue in developing a sampling plan is to specify the characteristics of those individuals/things (i.e. customers, companies, stores) from whom or about whom info. is needed to...
  • HIV and Adolescents - ACT for Youth

    HIV and Adolescents - ACT for Youth

    The American Academy of Pediatrics has a web site called Healthy Children which provides also a lot of information for adolescents and even into young adulthood. Finally, the Young Men's and Young Women's Health web sites are extremely useful providing...
  • Where is Death Valley? - AL Bateen Geography

    Where is Death Valley? - AL Bateen Geography

    PHYSICAL GEOGRAPHY (natural) Death Valley. Mt. Whitney. Pacific Ocean. Mojave Desert. KEY. San Francisco. Los Angeles. Better maps will have a key (like above), so the map stays clear; add a title and a north arrow. What is scale? Scale...
  • Harvard Referencing

    Harvard Referencing

    Harvard Referencing. Top tips for references. Make a list as you go along. Note the author, title, date and publisher. For articles, note the title and the journal title. For chapters, note the title and the book title. For websites,...
  • Doctrinal Mastery

    Doctrinal Mastery

    A.S.K "By contrast, I'm aware of another missionary who, knowing her unconfessed sin from before her mission would surely cause her to be sent home early, made her own plan to work extra hard during her mission and confess to...
  • 1  30131  (Nomenclature)  28  25 2 cations -

    1 30131 (Nomenclature) 28 25 2 cations -

    Cl Name: Iron (II) chloride You try: CuCl Cr2O3 * * การเรียกชื่อเขียนสูตร สารประกอบไอออนิก อ.ศราวุทธ Writing formulas for Binary Ionic Compounds with Transition Metals The roman numeral in the compound name gives the oxidation number of the metal ion.