耳でたどれるチャート:スクリーンリーダー向けデータ可視化の新しい設計

Charts You Can Navigate by Ear

Data Skeptic47分
耳でたどれるチャート:スクリーンリーダー向けデータ可視化の新しい設計

耳でたどれるチャート:スクリーンリーダー向けデータ可視化の新しい設計

Data Skeptic

0:0047:39

要約

Data Skeptic のKyleが、コロラド大学ボルダー校のJonathan Zong助教に、視覚障害・ロービジョンのユーザー向けにデータ可視化をアクセシブルにする研究を聞いた回。代替テキストの限界、盲目の共同研究者とのコデザイン、構造・ナビゲーション・説明という3つの設計次元、そのオープンソース実装Olliを取り上げる。触覚グラフィックスやマルチモーダル、今後の展望にも話が及ぶ。

この回のポイント

  • ●画像チャートは代替テキストが無いか役に立たないことが多く、単一の説明では読者の目的ごとの違いに対応しきれないとZong氏は述べている。
  • ●視覚障害のある研究者Daniel Hajas氏との約6か月のコデザインで、階層構造や「ズームする」感覚など、晴眼者の直感を覆す知見が得られた。
  • ●スクリーンリーダー向け可視化の設計次元として、構造(木構造など)、ナビゲーション、説明(内容・順序・詳細度)の3つを整理した。
  • ●OlliはJavaScriptのオープンソースライブラリで、Vega-Lite等の仕様や独自仕様からキーボードで操作できる木構造の説明を出力する。Altairのレンダラーもある。
  • ●13人の視覚障害・ロービジョン参加者の評価では、詳細への掘り下げや空間的な思考を歓迎する声と、表の方が良いという慎重な声の両方があった。
  • ●デフォルト設定の影響力や、表現的・美的な説明の可能性が今後の課題として挙げられた。

章立て

  1. 導入とゲスト紹介

    チャートが画像としてしか読み上げられない問題提起と、Jonathan Zong氏およびData and Design Groupの紹介。

  2. スクリーンリーダーと代替テキストの課題

    スクリーンリーダーの仕組みと、代替テキストが無い・役に立たない現状、可視化の説明に含めるべき内容の層について。

  3. 研究のきっかけとコデザイン

    パンデミック期の可視化の台頭を背景に、Daniel Hajas氏とのコデザインで得た気づきを語る。

  4. 触覚グラフィックスとマルチモーダル

    Tactile Vega-Liteなど触覚チャートの課題と、複数の感覚を組み合わせる方向性。

  5. 3つの設計次元:構造・ナビゲーション・説明

    木構造による階層化、空間的・ターゲット型ナビゲーション、説明の内容・順序・詳細度を解説。

  6. Olliの使い方と利用者の反応

    Olliの概要、既存チャートへの適用方法、13人の参加者による評価結果。

  7. 主観性・デフォルト・表現性

    可視化の客観性をめぐる議論と、デフォルト設定の力、美的な説明への期待。

  8. 今後の展望と参加の機会

    地図のアクセシビリティやWeb全般への展開、博士課程学生の募集、連絡先。

解説記事

Data Skeptic のHCI特集で、司会のKyleがコロラド大学ボルダー校のJonathan Zong助教(情報科学、Data and Design Group主宰)に話を聞いた。テーマは、視覚障害・ロービジョンのスクリーンリーダー利用者がチャートをどう体験できるか。論文で提案された設計の枠組みと、その実装であるOlliが中心となる。

「画像」としか読まれないチャート

スクリーンリーダーは画面上のテキストを音声で読み上げるソフトウェアで、macOSにはVoiceOverが組み込まれている。点字ディスプレイや拡大鏡と併用する人も多いという。画像は代替テキスト(alt text)で説明される必要があるが、Zong氏によれば、そもそも付いていないことが多く、「image」やファイル名だけが読まれることもある。かつてはSEO目的で無関係な語が詰め込まれた例もあったという。

善意で書かれた場合も課題は残る。可視化は、全体の傾向を見たい、特定の質問に答えたい、数値を探したいなど、読み方が多様だからだ。Zong氏は、何が有用な説明かは読者の目的に依存し、万能の解決策は作りにくいと述べている。説明に含める内容も、色や軸ラベルといった低レベルの要素から、統計的パターンや外れ値、さらに気候・選挙・COVIDのような社会的文脈まで層があると説明した。

当事者とのコデザイン

研究は、MITの可視化グループで博士課程にいた頃、パンデミックで「flatten the curve」のような可視化が重要になる一方、晴眼者以外を排除している点に気づいたことが出発点だった。Zong氏は、視覚障害のある研究者Daniel Hajas氏を共同設計者に迎え、約6か月、毎週Zoomで会合しながら試作を回した。Hajas氏は試作を試し、詳細なフィードバックをメールで返したという。

得られた知見の例として、読み上げ型インターフェースでは情報を線形に並べる必要があり、一覧表示だけでは全体像を把握しにくいことが挙がった。構造のある階層は、境界とドアのある「家」の中にいるような感覚を与える。階層を下りて詳細を増やす操作は、Hajas氏にとってデータに「ズームする」感覚だったという。Zong氏は、パンやズームといった探索のメタファーが暗黙に視覚的・空間的だったと気づいたと語る。「Nothing About Us Without Us」というスローガンを引き、社会的にも方法論的にも当事者の関与が重要だと強調した。

構造・ナビゲーション・説明

論文の貢献は、スクリーンリーダー向け可視化インターフェースの3つの設計次元だ。

  • 構造:チャートをどんなデータ構造でたどらせるか。500点の散布図を一覧で1点ずつ読むのは苦痛なので、木構造を採用した。根は全体の要約、その下に軸や凡例、さらに区間やカテゴリ、葉に個々のデータ点が並ぶ。興味のないチャートは素早く読み飛ばせる。
  • ナビゲーション:木構造に沿った移動(上下で階層、左右で兄弟ノード)に加え、上下をチャートの上下に対応させる空間的移動や、メニューで目的地へ飛ぶターゲット型移動を試作した。
  • 説明:内容(何を述べるか)、順序(「3つ中2つ目」のような位置情報を先頭に置くかカテゴリ名を先にするか)、詳細度(冗長か簡潔か)に分けた。初心者には丁寧に、慣れた人には短くという調整が可能で、続く研究ではこれらをユーザーがカスタマイズできるインターフェースも作ったという。

Olliと評価結果

OlliはNPMで導入できるJavaScriptのオープンソースライブラリだ。Vega-Liteなどの仕様、またはデータセットとフィールドの手動指定から、キーボードで操作できる木構造の説明を出力する。Pythonの可視化ライブラリAltair向けのレンダラーもある。既存のPNG画像でも、データとエンコーディングを記述したOlli仕様を並置すれば使える。Zong氏は、alt textでは作者の選択を受け入れるしかないが、Olliでは読者が自分で探索して結論を組み立てたり検証したりできると述べる。開発者にとっても、基本的なアクセシビリティの敷居を下げることを狙う。今後はデータジャーナリストの制作現場の制約を調べる研究も予定している。

13人の視覚障害・ロービジョンの参加者を対象にした評価では、「高いレベルから掘り下げられるのがよい」「空間的に考えられる」という声があった。一方で「表に戻してくれないと扱いにくい」という慎重な意見もあった。毎日使う人は独自のワークフローを持っており、新しい操作の学習コストが見合わない場合もあるとZong氏は認める。同時に、スクリーンリーダーの世界は一貫した設計慣習が乏しく、再利用可能なツールキットがその整備に寄与しうると述べた。

主観性、デフォルト、表現性

「可視化は語りか、中立なデータ提示か」という問いに対し、Zong氏はAmy Fox氏らの研究を紹介した。内容情報を消したチャートでも、見る人はデザインから作者や使用ツールを推測する。客観的に見せようとしても受け手は主観を読み込むという。Olliのデフォルト構造や順序も同様に解釈へ影響しうるが、多くの人はデフォルトを変えないため、その決定は大きな力を持つと自省した。

また、スクリーンリーダーは長らく効率重視で、そこには職場で同じペースで働くという正当な理由があったとしつつ、アートや詩を楽しめるような表現性・美的側面に大きな余地があると述べる。「All Text Says Poetry」というプロジェクトも紹介された。

まとめ

本回の要点は、アクセシビリティを「代替テキストを書くか否か」から、読者が自分で探索できる対話的な体験へと広げる視点だ。当事者との共同設計から導かれた構造・ナビゲーション・説明の枠組みは、可視化に限らずUI設計全般に応用できる可能性がある。Zong氏は、Vega-Liteのような宣言的仕様の発想をWebアクセシビリティ(WCAGやARIA)に持ち込む構想も示した。日本でもデータジャーナリズムや社内ダッシュボードで画像チャートが多用されており、チャートを画像として公開する際に、データ構造を併せて提供する発想は実務上の参考になるだろう。

文字起こし(英語・自動生成)

Thank you. Welcome to another installment of Data-Skeptic Human-Computer Interaction. I think most people take for granted how a chart works when you look at it as a glance, whether it's a bar chart or a line chart or a scatter plot. Take in the data, check out the axes, look for any indicators the chart creator put in there. Charts are a very visual language, and as the old adage says, a picture can be worth a thousand words.

But now put yourself in the shoes of someone that needs or chooses to use a screen reader. Imagine this, you're scrolling down a website, reading an article, it refers to some data in a chart, and in most cases, that chart will be announced as nothing more than an image or saying its file name. Those of you familiar with HTML real deep probably know about the alt tag, a well-intentioned tool to provide a description of what an image is, often rarely filled out with anything useful. But even if someone takes the time to write a good alt tag for a chart, you're only getting that one perspective the author chose to tell you. When I look at a chart, I often see different things than other people. My guest today is Jonathan Zong, an assistant professor of information science at the University of Colorado Boulder, where he leads the Data and Design Group. We talk about his paper, rich screen reader experiences for accessible data visualization. Jonathan and his collaborators have defined a rather novel system. It opens the door to finally provide people who need or choose to use a screen reader

with a much better interactive experience of data in a graph itself. All that and more coming right up. I'm Jonathan Zong. I'm currently an assistant professor of information science at the University of Colorado Boulder, where I lead a research group called the Data and Design Group. I'm also a faculty associate at the Berkman Klein Center for Internet and Society at Harvard University. And can you share some details on the Data and Design Group? The Data and Design Group is a research group that I started about a year ago at CU Boulder. And our interest is in kind of bringing an interdisciplinary perspective into understanding how the design of technology impacts how people experience it and how technology works in society. And a lot of our work has focused on accessibility in data visualization for blind and low vision screen reader users, where we're working with blind and low vision researchers and screen reader users to work together to design tools for working with data non-visually.

And we also do research on things like data ethics and the way that people kind of respond to unwanted data collection and how people are kind of reacting to new changes in technology that are brought about by things like AI and things that are kind of rapidly changing the way that data collection happens and making it more pervasive in society. and you mentioned it being an interdisciplinary group. Can you share a few details on what areas of scholarship or what field you draw from? So I think, you know, for a lot of these really important questions that people are facing with technology and society, we really need multiple kind of approaches to fully understand a lot of these problems. And HCI is kind of already an inherently interdisciplinary field. I'm coming from a background where I studied computer science, and I also have a background in visual arts and graphic design. I mean, I think that kind of naturally led me to some of my interests in data visualization. But I think we also have a lot to learn from people who are

really good at understanding, you know, kind of the history of how technologies are proliferated in society, right? So these are fields like science and technology studies and anthropology and things like that. Well, I haven't surveyed the listenership for this question, but I would guess most of them are sighted and not taking advantage of a lot of the accessibility features on their computers. So would you mind for what I presume is most listeners sharing a few details on maybe what are the challenges or what are the solutions available for people who need accessibility options? Most blind and low vision people who are using computers are using something called a screen reader, which is an assistive software that you can either download or sometimes It's built into operating systems. So for example, macOS has a screen reader called VoiceOver built in. And this is a program that kind of reads out content on the screen as text to speech. So there's a cursor that a user is controlling with keyboard commands, and it's kind of reading

out whatever is highlighted with the cursor. And of course, many people are using multiple assistive technologies. So there's also screen magnifiers for people who have partial vision. There's also braille displays that people can plug in and are compatible with screen readers so that the text that would be read out of speech is also displayed tactilely. And so when we think about, you know, the way that screen readers work, it's reading textual content aloud. And so when you encounter image-based content, for example, whether that's, you know, memes you share on social media or data visualizations, which are kind of more complex images in some ways. The person who, you know, created the website or posted the post usually has to provide something that's called an alt text or alternative text, which is a description of the image that can be read out by the screen reader. And when that's not provided, usually, you know, you get something like image, image, or like a file name, and none of that's really informative for someone who's

trying to kind of have a comparable experience of someone who's cited and looking at an image. Well, broadly speaking, how often is that alt text provided? And what's provided is something useful to a person who wants the content? Yeah, that's a great question. Because I think, as you say, there's kind of the baseline. Is there a description? Which often there isn't. And, you know, I think even though some platforms are getting better about reminding people to add alt text to images that they share, a lot of the time those are kind of skipped over. And even when there is a description, it's not always useful, right? I know like for a while, not so much anymore, but like there used to be kind of SEO hacking, right, where people would put information in the alt text that was meant to boost the search relevancy of whatever page it was on, but it wasn't actually describing the image, right? And so that would be like a very negative experience of an alt text. But sometimes, you know, even when people are well-intentioned, like thinking about accessibility, it's hard to know what information would be useful in an alt text, right?

And I think part of it is there's a lot of guidance out there, but it's just kind of hard to write a single description that fits every situation, especially when you consider something like a data visualization. People read data visualizations in a lot of different ways. You might be looking for a high level pattern in the data. You might have a specific question that you're trying to answer. You might be just kind of trying to look up a specific number. Right. So I think what makes a description useful also depends on what a reader is trying to accomplish, which makes it very hard to kind of create these one size fits all accessibility solutions, which is part of the challenge. Well, I'm a big fan of a guy named Edward Tufte, who you might know, but I'll share a few details for listeners who don't. And you can chime in as well if you'd like. One of his key concepts is the idea of chart junk. We want, if we're going to do a visualization, really boil it down to we want the reader to understand something from this. We want them to get, I don't know, the moral of the story or a sense of the distribution.

But I could also imagine where modern, like, image recognition technology might look at an image and say it's a bar graph with red bars, which tells me, frankly, nothing about it. You know, what are, I guess, some of the open challenges in getting things to work specifically in the data visualization world? Just to kind of like riff on, you know, some of Tufti's ideas a little bit, like I, you know, learned a lot from Tufti's books when I was learning data visualization. Right. And I think he's been very influential in the field. But I think also more recently, like people in the Viz community have kind of revisited some of those ideas. Right. Like chart junk. You know, I think a lot of artists and data journalists have also been exploring how a lot of these decorative elements can actually enhance someone's understanding of what data is trying to communicate. There's a lot of information about like the subject matter of data visualization, for example, that can be conveyed through just graphical presentation of data beyond like, is it just the simplest bar with like, you know, a simple color scale. So I think when we think about data visualization as communication, right, it's usually situated within, you know, you might find a data visualization in a news article or in a PowerPoint someone's giving, right?

So there are a lot of things that are being communicated that's not just kind of raw numbers. And I think the way that translates to accessibility, too, is like my colleague Alan did a project where he was classifying all the different levels of content that when you write a description of a data visualization, what about the chart are you actually describing? And there's, you know, low level features like, oh, like what color is the bar, right? Like the example you just gave. There's things like what are the access labels and things like that. But there's also higher level things like are there statistical patterns or outliers or are there kind of perceptual things happening? Like, I don't know if you've ever seen the updated version of the Anscombe Quartet, Datasaurus Dozen. Do you know what I'm talking about? I don't know the Datasaurus Dozen, but I know Anscombe's Quartet. Yeah, it's a similar idea. So it comes for Ted, like the idea that a data set could have like the same summary statistics, but visually be in very different shapes. The data source doesn't, it's just kind of another variation of this.

But I don't know, it's funny and interesting because one of the visualizations is like the shape of a dinosaur. But, you know, the summary statistics are kind of normal. And, you know, for something like that, right, like if you're only focused on describing the data, you might kind of miss out on like, actually, it kind of matters that this chart is in the shape of a dinosaur. And then finally, you know, there's an even higher level of current events like social and political context. Right. And especially with the way that data visualizations are kind of increasingly used to communicate really important information. Right. About climate or elections or COVID. Right. I think part of the challenge is like when we as people are writing these descriptions, do we have a clear sense of like who our audience is, what the message is that we're trying to convey, how things outside the visualization might be supporting or not supporting that message. Right. And, you know, that's before even we get to kind of how systems might automate something like this. So it seems like a surprisingly difficult problem then to maybe like infer missing alt text for a data visualization.

I'm sure we'll unpack that a little bit, but I guess I'm curious, and as best I can tell, you're a fully sighted person. How did you first take an interest in this challenge? I usually say, like, I think it's kind of right people right place right time I talk about maybe the time first I was doing my PhD at MIT in the MIT visualization group advised by Arvind Satyarayan And we were working on kind of tools for creating interactive visualizations and, you know, thinking about how to communicate data more effectively. And then around the year 2020, pandemic happens. And, you know, we were noticing like it felt like kind of data visualization was having this big moment, right, of being really central to how people were communicating around the pandemic. Like if you think of flatten the curve, this is probably like one of the most well-known data visualizations, maybe most consequential recent visualizations, right? And we were kind of noticing the way that as we were relying more on visualizations to really,

you know, cleanly and effectively communicate some of these ideas, we were also kind of experiencing this limitation of visualization, right? Which is that it's excluding people who aren't sighted. And, you know, I think there's this interesting tension that we were noticing. And around the same time, I was kind of collaborating with my lab mates and friends, Alan Lungard and Crystal Lee. And we had recently met my now friend and longtime collaborator, Daniel Hayash, who is blind and is currently a researcher at University College London. And we all kind of just got to talking and we were thinking about, you know, some of these issues around accessibility that were specific to data visualizations, right? Some of the things I was just talking about, about why data visualizations might be, you know, because they're read differently often from other types of images like photographs or whatever, where we might be able to kind of think about, you know, revisiting how can we create, you know, better, more effectively communicative, maybe more comparable experiences for screen reader users to the ones that sighted

people often enjoy. And yeah, what this kind of turned into was this, I think, like six-month-long process where we were all kind of meeting on Zoom every week, and we were engaging in this method called a co-design with Daniel as the co-designer who was bringing not only expertise about assistive technology and kind of the experience of being a screen reader expert, using one every day, but also, yeah, more broadly, that lived experience of disability, right, and that understanding of what issues would be really important for screen reader users. And I think this was a really transformative experience in my research career, because I think, you know, we were like brainstorming and Zoom, and then like we would pass around some prototypes of, you know, screen reader interfaces that we were making. And then like Daniel would try them, and then he would send back these really like detailed emails, a lot of notes. And that was so valuable because as a sighted person, you know, Maybe I bring a lot of expertise in interface design and data visualization, but I don't have that experience of being a screen reader user and living as a blind person, right?

So often when Daniel would kind of respond with all this feedback on prototypes, there would be a lot of things that would kind of really overturn my intuitions and also just kind of have really interesting kind of juxtapositions with kind of like conventional wisdom in HCI, right? because the theories and the frameworks we learn are also very primarily centered on graphical user interfaces. So I think for me, it was that experience of kind of being really clear about what expertise I did and didn't have and kind of bringing that intellectual humility into a collaboration was really productive, I think, for the work. And Daniel and I have been collaborating on a lot of the follow-up work to the study that we'll probably spend the most time talking about in this conversation. But yeah, I think, you know, there's a disability activism slogan called Nothing About Us Without Us. I think that kind of encapsulates the both like the kind of social importance of having people with experiences of disability involved in the design of the things that they will then use.

And also just kind of the methodological importance of like being able to validate your designs and your, you know, hypotheses with someone who would know. Are there any key insights from that co-design process you'd want to highlight? You know, one of the things that really struck me was like the way that Daniel talked about like some of the experiential aspects of using some of the prototypes that we were developing. So, for example, when you're looking at, you know, a graphical interface, you can kind of see all the things that exist on the page, right, and are available. And then you can kind of like get this really quick overview of everything and then maybe you know pick things to focus on based on the information hierarchy right like if you're on a page with like a very clear header structure you can kind of like skim the headers and then like find the thing that you actually want to focus on and things like that and this like overview detail idea is often not present in a lot of screen reader interfaces at least by default where because things are being read aloud there's kind of

implicitly you have to like linearize the interface into like an order that you can kind of arrow key navigate through so you know in some of our prototypes for example we would try these different like ways of organizing things we would try more like hierarchical structures like more like a tree than a list and daniel had really interesting observations about like when you have like a finite list of something right he kind of compared it to being in a house where like you are in like this bounded room and you kind of have these boundaries and know where the doors are. And like, so you can kind of like mentally divide the space into like meaningful sub areas. Whereas, you know, if you just kind of gave everything in like an unstructured list, you would have to do more work to build that model in your head, but not have it kind of be reflected in the interface. And then as another example, we were testing out some of these interactions where you would maybe like navigate to a different part of the chart and then you would maybe press the down arrow to kind of increase the level of detail and it would break down the part of the chart that you were

highlighting into maybe like sub intervals along the axis or something like that for example. And I remember you know in the paper Daniel wrote a bit about how this felt kind of in his head metaphorically like he was zooming into the data which I thought was really interesting because it made me think about how a lot of our kind of interaction, like data exploration metaphors, like panning and zooming are kind of implicitly like, if not visual, at the very least, like very spatial. So kind of being able to give some of those affordances of like spatiality in these screen reader interfaces, I think was a really cool thing to discover. Well, I think we're getting close to the design dimensions that are one of the main contributions of the paper. I want to get to those. But I guess before we delve right in, I know you're working on really the screen reader technology, but just out of curiosity, is there any interest maybe in other academics or in the community or with Daniel with people like that in something more tactile? You know, could there be a graph that I'm touching? Yeah, and we actually have done some follow-up work on tactile graphics.

we made this prototype system called Tactile Vega Light, which is an extension of Vega Light, which is a common visualization toolkit in JavaScript. And maybe some people know the kind of Python equivalent, which is Altair. And yeah, with tactile graphics, it's kind of this whole other thing, right? Where I think a lot of the theme of the work in accessibility is this idea of modality differences. The idea that just our senses kind of all work differently from each other, right? And the challenge in tactile charts is kind of, you know, we're thinking about either charts that are printed out on embossed paper with an embosser, or, you know, some people are 3D printing tactile charts, or there's also kind of these new refreshable tactile displays where it's kind of a pin array, and the pins kind of raise and lower programmatically, and you can convey pictures that way. The resolution is really low, and like some of the refresh rates are kind of slow so they're getting there and I think part of the challenge is like the way that we see things

is just very different from the way that we you know understand things through touch I saw this metaphor someone made of like using a tactile map for example is kind of akin to like if a sighted person were like looking through a small tube at a map right because you're kind of feeling locally with your hand or hands. And so you're kind of piecing together the big picture from like the parts, right? Whereas visualization, you often kind of get the big picture, at least implicitly, like more immediately, and you can kind of break it down into parts. And so with Tactile Vega Light, we were kind of thinking about, you know, how do we help people create tactile charts more easily? And we were doing a lot of research on some of the best practices that folks like the Braille Authority of North America have put out, like people who publish tactile books. And it turns out that even when you follow kind of all of the best practice guidelines, you can create these charts, but there's still more or less visual forms that have just been kind of printed out tactilely, right? And when you think about a bar chart, for example, bar charts are very useful

because like we have visual perception that's good at perceiving like relative length against a baseline or like things like area, right? But like tactilely, if you're feeling a bar chart, really what you're interested in is just like the ends of the bar, right? And so there's all this like space within the bar that does something visually, but isn't actually not that informative tactilely. So I think there's a lot of exciting work to think about, you know, how do we create these tactile first experiences of data? But yeah, lots of people are working on this, not just me. So I know like, for example, Kim Marriott at Monash is doing a lot of work with like refreshable tactile displays. Zhuyang So at Illinois has done a lot of cool work with braille displays and data, right? Yeah, I think a lot of the work that we've done with screen reader interfaces kind of also gives us a starting point to like think about maybe analogies to other modalities. And I've also done some work with multimodal systems that kind of combine like visualization and sonification and text.

And because all these senses are kind of work differently and are kind of doing different work. I'm increasingly of the opinion that like multimodal systems are kind of the way to go because they just they kind of just give everyone more options for understanding the data in different ways, you know, and even if you're fully sighted, a lot of people have told me that they also like having, you know, text or like something to listen to alongside the chart, right? Well, feel free to correct me if I'm wrong, but I guess in my reading, the paper's core contributions in terms of the design dimensions are those three, the structure, the navigation, and the description. Could you give some loose definitions on those and how they work towards your goal? So the goal of this project, you know, we set out to kind of think about what would screen reader experiences of visualizations be like that were kind of more comparable to the way that sighted readers are reading visualizations. We ended up framing our contribution as a set of design dimensions of screen reader interfaces for visualization. We were trying to think about, what are the things that a designer would have to consider

if they were making any type of data visualization interface? So we made a bunch of prototypes, and we were kind of almost trying to explore this design space of all the different ways that you could make screen reader versions of charts. We settled on a few that we liked the most and kind of built those into OLLI, but we recognized that there's a lot of different ways to do this. So what we were trying to do was create this framework that would be helpful to people who are trying to think about this systematically so that they can kind of make decisions about what appropriate for their system And so our three design dimensions that we identified through this prototyping process and this process of you know co and reflecting on our prototypes and exploring different variations, our design dimensions were structure, navigation, and description. I can kind of talk about each of these one at a time. So starting with structure, by structure we mean, you know, how do you represent a data visualization in a way that can be traversed by a screen reader. I mentioned before, right, there's this idea that you have to kind of like linearize things

into things that are like selectable by a cursor and then there will be a description readout, right? So we were kind of thinking about structure as like, yeah, like what is the data structure that a screen reader user is kind of traversing? You can arrange things as a list, but that might be really annoying because, you know, you might like if your scatterplot has like 500 dots on it and you have to go through those one by one in the list, like that's pretty frustrating. And so we explored other kinds of structures like trees and graphs. With trees, we could add some hierarchy so we could group things together. Part of it is also deciding what the nodes represent. Where we ended up, for example, was maybe there's a root node that's kind of an overall summary of what the chart is. So is this a scatterplot or a bar chart or what is it? And what are the axes and things like that? So for example, if someone is running into this on a web page that's like a news article or something, and they see this chart, they can kind of get an idea of what it is through like a very quick overview. And if they're like, whatever, I don't care about this chart, they can just go to the

next thing, right? They don't have to like go through every single scatterplot data point just to get past this chart. But if you want to go kind of down one level in the tree hierarchy structure, then we had nodes that represented the axes and legends. So you could get more information about the x-axis is this field. It has values from this to this. And then you could break those even further down into, okay, this has like these categories or these are the intervals along this axis. And then finally, at the very bottom level, like, you know, the leaf nodes of the tree, you could see the individual data points. And so this gives people more of like an option to kind of control what level of detail that they're exploring at. And also, you know, maybe like skip over parts of the chart that they're not as interested in so that they can save time. So that was structure. Second dimension is navigation. There's the obvious way, which is you know, moving along the structure itself. So like for the tree we had up was like up levels of the tree and down was down level of the tree. And then like left and right is like siblings on

the same level. But there are also other ways we could imagine. So for example, spatial navigation, if maybe up and down mean like towards the top and bottom of the chart rather than up and down the abstract structure, right? Maybe for scatterplot, that might be more interesting and important so that people can kind of understand, you know, spatial layout of the data and how it's presented visually. And then, you know, we were also thinking about, oh, like maybe people kind of already know that there's a thing that they're looking for in the chart and they just want to find it, but they don't want to figure out like what sequence of arrow keys gets them there, right? They don't have to like search for it. So we prototyped something we were calling targeted navigation, where you kind of just use menus to kind of directly jump to a place. And then maybe you can explore locally with the arrow keys once you've got there. Finally, we were thinking about description and we kind of broke this down into three parts. One is the content. So I was talking a bit about this earlier, but what is it about the chart that you're actually describing, whether

that's low level features like encodings and colors and stuff like that, or higher level features like the context of a chart. Like if you have a stock chart that includes 2008, you might want to say something about the financial crisis, right? Yeah. So, so there's, what are you actually describing? There's kind of what order are you putting the information in? Because in addition to data about the chart, there's probably also kind of like useful navigation and wayfinding related information that you might want to have. So for example, if you have a list of categories, right, it might be helpful to have like one of two, two of two, right? And, you know, often this, this type of thing is built into screen readers, but not always, or sometimes people have it turned off. But also, depending on what you're trying to do, if you're trying to find the third item in a list, having the index at the front is probably very useful. But if you're trying to find a specific category, for example, if you have to listen to 107, 207, 307, and you're just really just trying

to listen for what the category is, maybe it's better for the category to be upfront. Right. And then finally, we were thinking about verbosity of descriptions. So how much, how many words are you actually using to convey an idea? Right. So for someone who's maybe very new to navigating a system like this or new to data visualizations, maybe the description could be like access with values from 50 to 70. And then for someone who's like use the system a bunch of times, kind of knows a lot about what they're expecting each description to say. Right. It could just be like access 50 to 70 and that would be equally fine right for that user so we did some follow-up work to kind of developing interfaces for people to actually customize these descriptions and like kind of change what information is actually included change the ordering of things change the verbosity level i think that helps enable kind of the flexibility to adapt a description to what a reader or data analyst right is actually trying to do with the data well i'm hoping we can zoom in on Oli a little bit. Can you remind the users what it is and maybe comment on who the

audience or the users of it are? Oli is the open source library that we created kind of based on some of the things that we found in our research and some of our prototypes. And it's a JavaScript library that you can kind of, you know, NPM install and then you can include it in a web page. And you can basically either give it an existing visualization specification that's in an existing format like Vega Lite, or you can kind of manually specify in Oli, you know, a data set and fields and things like that. And it will output a keyboard navigable kind of tree-like structure with all these descriptions at varying levels of detail and kind of convenient key bindings for people to move around. And I think, you know, the audience, there's of course like the developer audience and the user audience. And, you know, we're really intending these to be useful for screen reader users who want to kind of explore data on their own, right? And, you know, when you have an alt text, you kind of just have to, you know, accept the choices that whoever wrote the alt text made

about what information was important to include, maybe what conclusions they decided to include or not include. Often, you know, people who are writing alt text make choices about like what level of detail to convey the information at, because you're always trading off like, how long is this? Like, how much do I have to listen to versus kind of getting the point across? And so with Oli, we're kind of hoping that it's helpful for screen reader users who kind of want to do that independent, like self-guided data exploration where you can, you know, just like a sighted reader visualization would maybe looking, look at the overarching patterns, drill down into individual parts of the visualization, right? And then kind of piece together conclusions for yourself, or at least verify, you know, things that are being described using the underlying data. And then on the developer side, yeah, I think, you know, we're hoping that it's kind of lowering the barrier to create baseline accessible experiences, right? Because accessibility is hard. Like, I recognize that, you know, I had to do a whole PhD to kind of know how to, like, you know, write a good alt text or whatever, right?

And so I'm very, like, accessibility is very important. And, like, we can't make excuses for, like, not making things accessible. But I also recognize that it's super hard, right? So I think that's why building toolkits like this that people can kind of drop in can, I think, maybe raise the quality of the baseline experience of visualizations. And we have this JavaScript library. We've also built Oli as a renderer in Altair, actually. So if you're a Python person, you can check out how to write Altair programs, how I'll put an Oli output in addition to a visualization. So I'm kind of wondering if, you know, let's imagine I was a software developer at like an online newspaper and we do a lot of charts, but they're, you know, rasterized charts. Yet I want to make them more accessible. Can I just NPM install OLLI and then, you know, put OLLI tags around my images and everything's fixed? Or do I go through and then do some sort of process of annotating every image so it has the OLLI format?

So if you have existing charts that you made either in a different visualization library that's not Vigalite or you just have an image that, you know, you have a designer made it in Illustrator, right? You can still use OLLI. One is you can just write an OLLI specification where you include the data set, you kind of include the encodings, right, where you map the fields to, you know, what would be the visual properties in your chart or even just kind of generic axes and legends. And, you know, you can add additional information about how certain things ought to be described. And that will generate an OLLI output kind of without, you know, an input visualization. So you can put that parallel to your existing PNG or whatever, or in addition to any alt text you may have already written. If there are other existing libraries that you use to make visualizations, I mentioned earlier that OLLI is extensible. And so I think part of what I'm hoping to do in the near future is actually do some more

research where we're collaborating with folks like data journalists and trying to understand their existing workflows and what kind of challenges they face with accessibility and maybe also just like tech infrastructure, right? Like what does it take to get a package approved for use in your publication or whatever? And yeah, think about kind of bringing these tools closer to where people are actually facing some of these challenges of like, we're on a deadline, but we need to make things accessible. And, you know, we know it's important. And yet, like the resources aren't always there, right? So I think, yeah, the more we can think about kind of those real world sites of use, the better it will be. Can you share some details on the feedback you got from participants in your study when they got to experience Ali and its outputs firsthand? When we were doing this research, we, you know, we were doing co-design with Daniel, but of course, you know, we knew that we couldn't just design things amongst ourselves without making sure that we were making things that worked for more people and weren't just kind of reflecting the opinions of people within our group, right? So we ended up doing this user study evaluation

with 13 blind and low vision participants, where we took a couple example prototypes that worked in different ways and showed them to people to kind of compare them with both their baseline experience of the ideal accessible visualization that has an alt text and maybe has the data table attached versus some of our prototypes. And we weren't really trying to see if it was better, for what definition of better, right? But I think we were really trying to understand how some of the choices that we were exploring in the prototypes were affecting the way that people thought about the data, right? So we were kind of trying to understand the influence of our design decisions on people's experience and understanding of the data. And I think we heard a lot of interesting things. So people said things like, I really like having the ability to scroll through at a higher level and then drill down deeper if that's of interest. So that was kind of telling us that our idea of adding some hierarchy and this idea of multiple levels of detail was pretty helpful for people to have that kind of drill down exploration strategy that you know a lot of sighted readers of visualizations are also using

And some people said things like, you know, it gives me different ways of thinking. I'm thinking more in spatial terms. Right. And I think that's also another advantage of having something you can kind of interact with and navigate around rather than a static alt text that just kind of blasts the paragraph at you is that we can start to develop some of those spatial metaphors, kind of like Daniel was saying with like the zooming metaphor. But I think we also, you know, very understandably heard things from people that were kind of more cautious, right? Someone was like, you know, I think this might be understandable to someone who's done this before, but I don't, you know, really do well with these charts unless they're converted back into tables. And what that reflects is that people who are using screen readers every day have very kind of personalized workflows that they're used to. And there's also kind of conventions and, you know, what common practices are that we have to kind of understand, right? And so we can imagine maybe a situation where, you know, you have to put up a lot of upfront effort to learn how to use an interface like

Oli. Maybe after you do that, it could be worth it for like an advanced user who really wants to understand the data. But also maybe for many people, it's like not worth that effort, right? And I think that's a totally understandable reaction for someone to have. So I think, you know, we can think about how to make these things more learnable, obviously. But I think there's a broader point about, you know, we have very established user experience conventions for graphical interfaces. There's like common sets of components and most websites kind of have one of a few pretty standard layouts. Right. But with screen reader interfaces, there's so much chaos. Right. I don't think that users have really come to expect like a consistent experience from app to app or like website to website. And so I think that's another reason why reusable toolkits like Ali that people can kind of drop into their own websites could make a difference is kind of establishing more of that consistent design language around screen reader interfaces. So I know there are different perspectives in the data visualization world. world. I'll maybe, I don't know, steel man or straw man, two of them, but one is to say that

every choice I make from my axis and my line thickness is a decision I make and I'm narrating my chart. And maybe there's another that says I'm really just trying to give you the unopinionated data and you were to draw your own conclusions from it. Do philosophies like this get involved in Ali in any way? Absolutely. I think that's a really good kind of encapsulation of the two polls of this conversation. I think increasingly, I'm seeing a lot of really cool work. My former colleague, Amy Fox, who's at UCSD now, did really interesting work, for example, where she did a study where they basically took a bunch of charts from a bunch of different places on the internet, made in a bunch of different tools, like some from Reddit, some from Tumblr, some from kind of more like data artist circles, right? Redacted all the information that was about what the data was about so like no titles no like labels or anything and then just showed people the charts even for the most simplest like chart junk free visualizations people were instantly kind of

looking at the graphic design of the chart and like making assumptions about the person who made the chart or like what tools they used right for example i don't know oh this looks like it's kind of the default settings in matplotlib so like a scientist must have made this but like we have no And then vice versa, there's the work of folks like Nadia Bremer and like Stephanie Pasovich, right, who illustrate these really like evocative and kind of communicative data visualizations that are very stylized, but very effective. Like they've won a lot of like journalism awards, right? And then, you know, with all the information redacted, people will be like, oh, like, it looks like someone hand drew this. So they probably didn't know what they were doing or whatever. Right. So I guess what I'm trying to say is that I think regardless of whether or not we think we're objectively conveying the data on the receiver end, like when people read things, they will bring their own subjectivity to it. Right. And kind of impute subjectivity onto you if you don't intend it right through just the choices that you make.

Yeah, your question about how this translates to accessibility and Oli is like really interesting. And yeah, I would love to think about this more. But I think my initial instinct is like, yeah, we have made this kind of very templatized, standardized tool. Right. And I think we've made a lot of default decisions about like what the structure should be and like what, you know, the default orders of descriptions should be and like what should be included. And I think defaults are so powerful. Right. We've given people the ability to customize things, but just based on what we know about how things work in HCI, most people will never touch the defaults. And so I think there's an element of, yeah, those decisions have a lot of power over interpretation and the way that people kind of perceive the like authoritativeness or like objectivity of something. I also think that, you know, I would love to think more about how do we make accessible the more expressive parts of data visualization, right? you have this really intricately illustrated visualization that, you know, was handcrafted by an artist versus a plot that is the default matplotlib output. Should those descriptions be

the same, you know? And I think there's a broader idea in accessibility that, like, for so long, screen reader interfaces have been very purely efficiency focused. I think there's a very good reason for that. You know, a lot of what the BLV community felt was very important is, you know, being able to like be part of conversations in the workplace and like, you know, be on a team and be able to do their work at the same pace as anyone else. Right. And with the same efficiency. But I think what that means is there's like a huge opportunity to think about kind of the expressiveness and like the aesthetics of screen reader interfaces. And, you know, how do we enjoy things like art and poetry, like through screen readers. Right. And I know there's lots of other folks thinking about this too there's a project called all text says poetry which is kind of as the name describes right kind of trying to get people to think about descriptions beyond kind of the utilitarian aspect of it and yeah i just think that's that's a very rich area to think about kind

of you know before we mentioned the levels of like is there even a description is the description useful i think the level after that is is the description like enjoyable to read right i'll Just say that if people are interested in learning more, you know, my research group's website has some of our more recent work that's kind of built on Oli. So I mentioned before, like the customizable description projects, the tactile charts project. We've also done some work with how do we make tools that enable blind and low vision users to create their own visualizations and not just kind of consume visualizations made by other people. More recently, we've done some work kind of collaborating with an oceanography lab led by a blind scientist where we were kind of looking at the ways that they use kind of multi-sensory data representations like charts and descriptions and like, you know, 3D printed tactile maps and things like that to kind of collaborate and do science. Right. And how those representations kind of facilitate communication. So, yeah, I think there's like a whole wide world out there in data visualization accessibility.

Well, OLLI is a relatively new project and growing project. Do you have any idea about aspirations you have for it in the next couple of years or maybe what a fully deployed project looks like where all of our accessibility needs are solved? Yeah, I mean, I'm still super committed to kind of like building on Oli, both as like a thing that's a useful instrument for us to do research on and kind of learn more about what's, you know, what are good experiences for screen reader users, but also just as a tool that people can use. So, you know, for example, some of the things we have coming down the pipe, there's a student that's working with me who's really interested in kind of accessible mapping, right? And thinking about how like geospatial visualizations pose a lot of unique challenges around what regions are adjacent to other regions and, you know, what kind of like politically and culturally defined ways of like grouping geography are and how we can apply, you know, similar approaches in OLLI to maps. I think longer term, I'm also hoping that the approach that we've kind of set out with in OLLI, that's really building off of Vega Lite and kind of other declarative specifications, is, you know, how do we kind of think about broadening this from data visualization and think about kind of improving the accessibility of like web applications in general?

And, you know, we have a lot of like existing very important standards around that. WCAG, the Web Content Accessibility Guidelines, and ARIA, which is the markup language where you annotate HTML with additional accessibility metadata. But I think the really nice thing that we've learned in the visualization community from grammar based approaches like VegaLite is that when you when you have declared a specification where you're kind of, you know, writing visualizations with high level concepts like like fields and encodings, rather than like rendering level concepts, like you might write in something imperative like D3, you get all these kind of nice properties. You have more kind of guarantees about like, if a program compiles, it'll also produce like a good output, right? You also get nice like compositional ideas where like if you put together two subcharts like in a layer or whatever, you just automatically get this other type of chart. And I think there's a lot we could learn from those almost like programming language design ideas

to this process of like making the web accessible because, you know, guidelines are guidelines, but for a developer who is trying to put these into practice, it's so like situational. It's so easy to mess up, right? And we're all kind of reinventing it from scratch every time because there's a lack of consistency. So yeah, I'm kind of longer term thinking about these ideas of structure, navigation, description, where visualization could, I don't know, maybe possibly be like this special case for us to study things that might be applicable more broadly to like graphical user interfaces in general. What are some opportunities for grad students or maybe an ambitious undergrad in that area? Ah, yes. Well, I am actively recruiting PhD students. And that could be, you know, if you're an undergrad who's really interested in doing this kind of human computer interaction research. Or that could be, you know, if you've like worked for a while and you kind of want to take the skills that you've developed in industry towards some of these problems that kind of

unfold over like a longer time, right, where we get to kind of go deep into questions and, you know, work with communities that, you know, you might not have time to work with in like a product cycle and industry, right. That's definitely, you know, an opportunity to work together. Yeah. Where can listeners follow you online? Yes. So these days, well, first of all, you can go to either my personal website, which is JonathanZong.com, or my research group's website, which is data-and-design.org to get updates about the work that we're doing. You can also find me on social media these days. I'm mostly on Blue Sky and LinkedIn, and I should just come up if you search my name. we'll have links to all the above in the show notes for listeners to follow up Jonathan thank you so much for taking the time to come on and share your work thank you so much yeah and thank you for the interesting questions

番組の概要欄(原文)

Kyle speaks with Jonathan Zong about making data visualizations accessible to blind and low-vision users through screen readers, interactive tools like Olli, and inclusive design approaches that enable independent data exploration.

出典・この記事について

番組
Data Skeptic(Kyle Polich)
元の回
Charts You Can Navigate by Ear
配信日
要約作成日
公式サイト
dataskeptic.com

この要約・解説記事は、元の音声を文字起こしした内容をもとに AI(Claude)が日本語で作成したものです。 固有名詞や数値に誤りを含む場合があるため、正確な内容は元の音声でご確認ください。制作方法について

X でシェアSpotify で聴くApple Podcasts で聴く

関連エピソード