Dell IEEE 802.11 TGT Output

January 2006 doc.:IEEE 802.11-05/0144 r0 Video Testing Methodology Authors: Name Company Address Phone email Philip Corriveau Intel HF3-96 5200 NE Elam Young Pkwy, Hillsboro, OR. 97124 (503)-696-1837 [email protected] Audrey Younkin Intel HF3-96 5200 NE Elam Young Pkwy, Hillsboro, OR. 97124 (503)-696-3947 [email protected]

Chris Olsen Intel HF3-96 5200 NE Elam Young Pkwy, Hillsboro, OR. 97124 (503)-696-7548 [email protected] Royce Fernald Intel HF3-96 5200 NE Elam Young Pkwy, Hillsboro, OR. 97124 (503)-696-4318 [email protected] Rik Logan Intel 5200 NE Elam Young Pkwy Hillsboro, OR 87124 (503)-712-1675 [email protected] Uriel Lemberger Intel PO Box 1659, Matam Industrial Park, Haifa 31015 Israel +972-4-865-5701

[email protected] Neeraj Sharma Intel 13290 Evening Creek Drive, San Diego, CA 92128 (858)-385-4112 [email protected] Sasha Tolpin Intel PO Box 1659, Matam Industrial Park, Haifa 31015 Israel +972-4-865-5430 [email protected] Notice: This document has been prepared to assist IEEE 802.11. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein. Release: The contributor grants a free, irrevocable license to the IEEE to incorporate material contained in this contribution, and any modifications thereof, in the creation of an IEEE Standards publication; to copyright in the IEEEs name any IEEE Standards publication even though it may include portions of this contribution; and at the IEEEs sole discretion to permit others to reproduce in whole or in part the resulting IEEE Standards publication. The contributor also acknowledges and accepts that this contribution may be made public by IEEE 802.11. Patent Policy and Procedures: The contributor is familiar with the IEEE 802 Patent Policy and Procedures , including the statement "IEEE standards may include the known use of patent(s), including patent applications, provided the IEEE receives assurance from the patent holder or applicant with respect to patents essential for compliance with both mandatory and optional portions of the standard." Early disclosure to the Working Group of patent information that might be relevant to the standard is essential to reduce the possibility for delays in the development process and increase the likelihood that the draft publication will be approved for publication. Please notify the Chair < [email protected]> as early as possible, in written or electronic form, if patented technology (or technology under patent application) might be incorporated into a draft standard being developed within the IEEE 802.11 Working Group. If you have questions, contact the IEEE Patent Committee Administrator at .

Submission Slide 1 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 2 Philip J. Corriveau - Intel January 2006

doc.:IEEE 802.11-05/0144 r0 Intel Video Testing Methodology Concepts and Methods Philip Corriveau & Audrey Younkin User Centered Design Media and Acoustics Perception Lab Submission Slide 3 Christopher Olsen & Royce Fernald Platform Systems Technology Wireless Integration Team Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda

Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 4 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Goals Quantify and characterize the end-user video over wireless experience using objective tools in order to reduce the need for expensive and time-consuming subjective video tests Objectively measure the playback performance of a set of degraded video clips with the Intel Video Gross Error Detector application Have end-users rate the degraded samples according to their perception of the video playback experience Correlate subjective evaluations with objective GED scores to establish a MOS (mean opinion score) scale Submission

Slide 5 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 6 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0

Video GED Overview The Video Gross Error Detector is a high-level video performance analysis application Provides objective data on dropped, repeated or out-ofsequence frames Enables automated, quantitative, repeatable measurements of playback smoothness and frame rate stability Other video quality analysis tools (such as VQM) are used in addition to the GED to measure image quality Submission Slide 7 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Platform Independence The GED supports any video playback solution that can be connected to a hardware capture device Since the GED operates directly on capture files, it can measure and compare the video performance of various network transports, operating systems, streaming applications, media players and compression formats The GED allows direct comparison between PC platforms and consumer electronics devices Submission Slide 8

Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 9 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 GED Methodology Source clips are encoded with GED frame identifiers, a

sequence of color blocks Video clips are played through the system under test and captured The GED reads the color blocks to locate dropped or repeated frames GED Encode Submission Slide 10 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 GED Processing 0. Source Material 1. Marked Source Material GED Encode Video Encoder 2. Compressed and Marked 3. Capture Results 4. GED Analysis System Under Test GED Decode Submission

Slide 11 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 12 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0

Subjective Assessment Methodology Video sequences with varying levels of degradation were generated to represent errors generally encountered during video over wireless playback A series of 240-frame Standard Definition (720x486) clips was assembled into a one hour test session (10 video clips x 5 conditions x 2 Error types) Fifty non-expert subjects (50% male and 50% female) were asked to evaluate each clip for playback smoothness and fluidity, not their opinion of the video content Before each test session, subjects were shown a sample of the best and worst clips to establish a frame of reference in order to reduce the impact of participant inherent biases Submission Slide 13 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Subjective Video Quality Scale After each video sequence, participants were presented with a choice of five adjectives describing their opinion of the video experience on a subjective scale Numerical Value

User Experience 5 Not Perceptible 4 Perceptible but Not Annoying 3 Perceptible & Slightly Annoying 2 Perceptible & Very Annoying 1 Perceptible & Extremely Annoying Trial Structure Grey Frame Trial 1 Test Video Clip Submission Grey Frame Vote

Grey Frame Trial 2 Test Video Clip Grey Frame Vote Slide 14 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Randomization The presentation order of the video content was randomized using a pseudo-random number generator tool to prevent ordering effects, i.e. to mask any tendency for a participant to rate a clip in relation to the previous one Randomization is a key element of psycho-visual testing that ensures participants do not see the material in a repeated fashion that would allow a learning effect Submission Slide 15 Philip J. Corriveau - Intel

January 2006 doc.:IEEE 802.11-05/0144 r0 Participants sat at a predetermined viewing distance 20 H 5H Submission Slide 16 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Subjective Assessment Flow Diagram Tutorial Shown Perceptible & Extremely Annoying Video Shown Not Perceptible Video Practice Trial Clips Randomized Subjective Vote

Submission Slide 17 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 18 Philip J. Corriveau - Intel January 2006

doc.:IEEE 802.11-05/0144 r0 Overall Means for Dropped and Repeated Errors C u r r e n t e ffe c t: F ( 1 , 4 8 ) = 6 .4 0 0 5 , p = .0 1 4 7 5 V e r tic a l b a r s d e n o te 0 .9 5 c o n fid e n c e in te r v a ls Im p e r c e p tib le = 5 P e r c e p tib le b u t N o t A n n o y in g = 4 P e r c e p tib le & S lig h tly A n n o y in g = 3 MOS P e r c e p tib le & V e r y A n n o y in g = 2 P e r c e p tib le & E x tr e m e ly A n n o y in g = 1 D ro p p e d F ra m e s R e p e a te d F r a m e s TYPE O F ER R O R Significant difference found between dropped frames and repeated frames. Submission Slide 19 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Collapsed Across Dropped and Repeated for Each Condition C u r r e n t e ffe c t: F ( 4 , 1 9 2 ) = 8 9 1 .0 9 , p = 0 .0 0 0 0 V e r tic a l b a r s d e n o te 0 .9 5 c o n fid e n c e in te r v a ls

Im p e r c e p tib le = 5 P e r c e p tib le b u t N o t A n n o y in g = 4 P e r c e p tib le & S lig h tly A n n o y in g = 3 MOS P e r c e p tib le & V e r y A n n o y in g = 2 P e r c e p tib le & E x tr e m e ly A n n o y in g = 1 1 10 50 80 200 E R R O R S ( C a te g o r ic a l) Plotted means for both error types. Submission Slide 20 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Collapsed Across Dropped and Repeated for Each Clip C u r r e n t e ffe c t: F ( 3 6 , 1 7 2 8 ) = 8 .4 7 9 0 , p = 0 .0 0 0 0 V e r tic a l b a r s d e n o te 0 .9 5 c o n fid e n c e in te r v a ls

Im p e r c e p tib le = 5 P e r c e p tib le b u t N o t A n n o y in g = 4 P e r c e p tib le & S lig h tly A n n o y in g = 3 MOS P e r c e p tib le & V e r y A n n o y in g = 2 P e r c e p tib le & E x tr e m e ly A n n o y in g = 1 0 1 10 50 80 200 E R R O R S ( C a te g o r ic a l) B a llo n B u ild in g M & C T ig e r L e tte r s W a te r fa ll F o o tb a ll S h ip S u s s ie F lo w e rs User ratings are content dependent.

Submission Slide 21 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 MOS Dropped Frames M e a n s o f D r o p p e d E r r o r s fo r a ll D iffe r e n t S e q u e n c e s U s e r E x p e r ie n c e = 4 .7 8 9 6 - 1 .3 6 5 7 * l o g 1 0 ( x ) Im p e r c e p tib le = 5 P e r c e p tib le b u t N o t A n n o y in g = 4 P e r c e p ti b l e & S lig h tly A n n o y i n g = 3 MOS P e r c e p tib le & V e r y A n n o y in g = 2 P e r c e p ti b l e & E x tr e m e ly A n n o y i n g = 1 0 20 40 60 80 100 120 140 160 180 200 D ro p p e d E rro rs

Log fit graph for predicting dropped errors. Submission Slide 22 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 MOS Repeated Frames M e a n s o f R e p e a t e d E r r o r s f o r a ll D if f e r e n t S e q u e n c e s U s e r E x p e r ie n c e R e p e a te d = 4 .5 7 7 6 - 1 .2 6 3 8 * lo g 1 0 ( x ) Im p e r c e p tib le = 5 P e r c e p tib le b u t N o t A n n o y in g = 4 P e r c e p tib le & S lig h tly A n n o y in g = 3 MOS P e r c e p tib le & V e r y A n n o y in g = 2 P e r c e p tib le & E x tr e m e ly A n n o y in g = 1 0 20 40 60 80 100 120 140 160 180 200 E rro rs

Log fit graph for predicting repeated errors. Submission Slide 23 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 MOS Combined (Dropped & Repeated frames) MOS Log fit for both Dropped and Repeated Errors 5 MOS 4 MOS Dropped & Repeated Log. (MOS Dropped & Repeated) 3 2 1 0 20 40

60 80 100 120 140 160 180 200 y = -0.571Ln(x) + 4.6836 2 R = 0.9859 Errors Log fit graph for predicting dropped and/or repeated errors with high correlation. Submission Slide 24 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Results & Analysis Dropped Frames Experience Errors Dropped & Repeated Frames Repeated Frames Percentage

Experience Errors Percentage Experience Errors Percentage 5 0.7 0.3% 5 0.5 0.2% 5 0.6 0.3% 4 3.8 1.6% 4

2.9 1.2% 4 3.3 1.3% 3 20.4 8.5% 3 17.7 7.4% 3 18.9 7.8% 2 110.3 46.0% 2

109.5 46.0% 2 109.9 46.0% 1 595.4 1 677.4 1 633.4 Based on 8 sec clip = 240 frames Submission Slide 25 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda

Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 26 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Next Steps Repeat non-expert subjective assessment with another sample group Increase the sample resolution near the perceptibility threshold (between 3 and 20 errors per clip) Correlate GED and subjective data to link (physical and mac) layer metrics for 802.11 Develop a video quality model for 802.11 Submission

Slide 27 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0 Agenda Goals Video Gross Error Detector Overview Gross Error Detector Methodology Subjective Assessment Methodology Results and Analysis Next Steps Summary Submission Slide 28 Philip J. Corriveau - Intel January 2006 doc.:IEEE 802.11-05/0144 r0

Summary The GED is a key tool for video over wireless performance analysis, providing objective data on dropped and repeated frames When correlated with a MOS (mean opinion score) scale through subjective testing, the GED can help characterize the expected end-user experience MOS data from objective tools saves significant time and money compared to subjective video analysis Submission Slide 29 Philip J. Corriveau - Intel

Recently Viewed Presentations

  • Vtnda Cmiyyti Tkilatlarnn Davamllq ndeksi 2011 Vtnda Cmiyyti

    Vtnda Cmiyyti Tkilatlarnn Davamllq ndeksi 2011 Vtnda Cmiyyti

    Yerli icra orqanları QHT-lərin xaricdən maliyyələșməsinə diqqət çəkirlər onların işinə məhdudiyyət qoyulur 2011-ci ilin fevralında Sahibkarlığın və Bazar İqtisadiyyatına yardım Fondu QHT Dəstək Şurasının sifarişi ilə hazırladığı "QHT-lərin məsuliyyət xartiyası"nı təqdim etdi Xartiyanın ...
  • Perspectives and Strategies for Supporting Algebra Success for

    Perspectives and Strategies for Supporting Algebra Success for

    Basic number sense, math facts, arithmetic. Support in transition from concrete to abstract, building conceptual understanding ... Teachers need to acquire a broader toolkit or "bag of tricks" to engage students, give strugglers different ways to understand concepts and solve...
  • THE TRINITY - WordPress.com

    THE TRINITY - WordPress.com

    II. Basic Terminology. 1.Economic Trinity - The study of the activity and redemptive outreach of the Triune God of Scripture to entities outside of Himself, especially the human populace.. 2. Hypostatic . Union - (ὑπόστασις in Greek = subtance or...
  • ADA Compliant Lecture PowerPoint

    ADA Compliant Lecture PowerPoint

    Neo-Piagetian theories: Operational thinking represents expansion of information-processing capacity. Central conceptual structures enable children to think effectively in wide range of situations. Neo-Piagetian approaches account for unevenness in cognitive development.
  • Light Blue Sky - Mrs. Rakers&#x27; Science Lab

    Light Blue Sky - Mrs. Rakers' Science Lab

    Interpreting and reading graphs. 2. Constructing data tables. Constructing different types of graphs (line graphs, bar graphs, circle graphs) ... Circle graphs are best used to give the viewer an overall or broad picture view of smaller groups of data...
  • User Manual Banking Correspondent/Facilitator exam ...

    User Manual Banking Correspondent/Facilitator exam ...

    Who can attempt/apply: Any one interested to become Banking correspondent or want to impart banking services. What is fee: 800+ GST for attempt number 1st, 3rd ,5th and on; 400+GST for attempt number 2nd , 4th , 6th and so...
  • CSCE 330 Programming Language Structures

    CSCE 330 Programming Language Structures

    Reality is not obtained by consensus. Reality must be achieved individually. Jean-Paul Sartre (1905-1980): the world is absurd (without meaning); what meaning there is only holds for the individual. Gabriel Marcel (1889-1973): the world is recognized by each individual in...
  • DNA Barcoding Ava Gansrow, Brielle Ohlsen, Laura Schillizzi,

    DNA Barcoding Ava Gansrow, Brielle Ohlsen, Laura Schillizzi,

    First collect you're specimen, then take a piece of sample tissue using tweezers. Put the tissue sample into a little tube and add 300 ul of lysis solution and grind the sample. Incubate for 10 minutes at 65C. After centrifuge...